Introduction to Computer Vision: Plant Seedlings Classification¶
Problem Statement¶
Context¶
In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
Objective¶
The aim of this project is to Build a Convolutional Neural Network to classify plant seedlings into their respective categories.
Data Dictionary¶
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
The dataset can be download from Olympus.
The data file names are:
- images.npy
- Labels.csv
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
- Black-grass
- Charlock
- Cleavers
- Common Chickweed
- Common Wheat
- Fat Hen
- Loose Silky-bent
- Maize
- Scentless Mayweed
- Shepherds Purse
- Small-flowered Cranesbill
- Sugar beet
Note: Please use GPU runtime on Google Colab to execute the code faster.¶
Importing necessary libraries¶
#print current env PATH variable value
!printenv PATH
/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin
#Update PATH environment variable
# Append /root/.local/bin to the existing path environment variable to get rid of these installation warnings - f2py, f2py3 and f2py3.10 are installed in '/root/.local/bin' which is not on PATH.
%env PATH=/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin:/root/.local/bin
env: PATH=/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin:/root/.local/bin
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
# Installing the libraries with the specified version.
# uncomment and run the following lines if Jupyter Notebook is being used
#!pip install tensorflow==2.13.0 scikit-learn==1.2.2 seaborn==0.11.1 matplotlib==3.3.4 numpy==1.24.3 pandas==1.5.2 opencv-python==4.8.0.76 -q --user
Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import cv2
import os
import tensorflow as tf
import time
import re
from tensorflow.keras import backend
from google.colab.patches import cv2_imshow
from sklearn.model_selection import train_test_split
from tensorflow.keras.models import Sequential # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam #Optmizer
#Imports functions for evaluating the performance of machine learning models
from sklearn.metrics import f1_score,accuracy_score, recall_score, precision_score, classification_report
from tensorflow.keras.callbacks import ReduceLROnPlateau
from tensorflow.keras.preprocessing.image import ImageDataGenerator
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
print(tf.__version__)
2.15.0
Loading the dataset¶
# Uncomment and run the below code if you are using google colab
from google.colab import drive
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
# Load the image file of the dataset
images = np.load('/content/drive/My Drive/PlantSeedlingClassification/images.npy')
# Load the labels file of the dataset
labels = pd.read_csv('/content/drive/My Drive/PlantSeedlingClassification/Labels.csv')
Data Overview¶
Understand the shape of the dataset¶
print('Shape of the images array: ', images.shape)
print('Shape of the labels array: ', labels.shape)
Shape of the images array: (4750, 128, 128, 3) Shape of the labels array: (4750, 1)
Exploratory Data Analysis¶
- EDA is an important part of any project involving data.
- It is important to investigate and understand the data better before building a model with it.
- A few questions have been mentioned below which will help you understand the data better.
- A thorough analysis of the data, in addition to the questions mentioned below, should be done.
- How are these different category plant images different from each other?
- Is the dataset provided an imbalance? (Check with using bar plots)
Plotting Images using opencv and matplotlib¶
cv2_imshow(images[8])
plt.imshow(images[8])
<matplotlib.image.AxesImage at 0x7ff8806a3b10>
- We can observe that the images are being shown in different colors when plotted with openCV and matplotlib as OpenCV reads images in BGR format and this shows that the given numpy arrays were generated from the original images using OpenCV.
- We will convert these BGR images to RGB images during Data Pre-Processing so we could interpret them easily.
Plotting randomly images of each category¶
#Count of uniques plant species/categories
labels.nunique()
| 0 | |
|---|---|
| Label | 12 |
- We observe that there are 12 different categories of Plant species in the data set
Plot unique categories images
def plot_images(images,labels):
num_classes=10 # Number of Classes
categories=np.unique(labels)
keys=dict(labels['Label']) # Obtaing the unique classes from y_train
rows = 3 # Defining number of rows=3
cols = 4 # Defining number of columns=4
displayed_list = []
fig = plt.figure(figsize=(10, 8)) # Defining the figure size to 10x8
for i in range(cols):
for j in range(rows):
if not displayed_list: #First entry :Displayed list is empty
random_index = np.random.randint(0, len(labels)) # Generating random indices from the data and plotting the images
ax = fig.add_subplot(rows, cols, i * rows + j + 1) # Adding subplots with 3 rows and 4 columns
displayed_list.append(keys[random_index])
ax.imshow(images[random_index, :]) # Plotting the image
ax.set_title(keys[random_index])
else :# If species is already displayed, find a new one
while (keys[random_index] in displayed_list and len(displayed_list) < len(keys)):
random_index = np.random.randint(0, len(labels)) # Generating random indices from the data and plotting the images
ax = fig.add_subplot(rows, cols, i * rows + j + 1) # Adding subplots with 3 rows and 4 columns
if (keys[random_index] not in displayed_list):
displayed_list.append(keys[random_index])
ax.imshow(images[random_index, :]) # Plotting the image
ax.set_title(keys[random_index])
plt.show()
plot_images(images,labels)
- Each plant categories shows difference in leaf shape,leaf arrangement,color,texture,growth habit like upright,grass like,erect bracnching stem,climbers.
Based on the research of these categories here are key features of each plant categories.
- Leaf Shape and Arrangement:
- Black-grass: Long, narrow leaves with parallel veins, arranged alternately on the stem.
- Charlock: Broad, oval-shaped leaves with toothed margins, arranged alternately.
- Cleavers: Narrow, lance-shaped leaves with a whorled arrangement (multiple leaves arising from the same point on the stem).
- Common Chickweed: Oval-shaped leaves with pointed tips, arranged oppositely on the stem.
- Common Wheat: Long, narrow leaves similar to Black-grass, but often with a slightly bluish-green color.
- Fat Hen: Diamond-shaped or triangular leaves with toothed margins, arranged alternately.
- Loose Silky-bent: Fine, grass-like leaves with a delicate appearance.
- Maize: Long, broad leaves with parallel veins, arranged alternately. They often have a distinct midrib.
- Scentless Mayweed: Finely divided, feathery leaves, giving it a fern-like appearance.
- Shepherds Purse: Basal rosette of deeply lobed leaves, with smaller, arrow-shaped leaves along the stem.
- Small-flowered Cranesbill: Rounded or kidney-shaped leaves with palmate lobes (like fingers on a hand).
- Sugar beet: Large, oval-shaped leaves with prominent veins, arranged alternately.
- Leaf Color and Texture:
- Black-grass: Dark green to slightly purplish.
- Charlock: Bright green with a slightly hairy texture.
- Cleavers: Light green with a slightly sticky texture.
- Common Chickweed: Light green and smooth.
Common Wheat: Bluish-green.
- Fat Hen: Dull green with a slightly mealy texture.
- Loose Silky-bent: Light green and very fine.
- Maize: Bright green with a prominent midrib.
- Scentless Mayweed: Dark green and finely divided.
- Shepherds Purse: Dark green.
- Small-flowered Cranesbill: Light to medium green, often with reddish stems.
- Sugar beet: Dark green and glossy.
- Growth Habit:
- Black-grass: Upright, grass-like growth.
- Charlock: Erect with branching stems.
- Cleavers: Sprawling or climbing.
- Common Chickweed: Low-growing and mat-forming.
- Common Wheat: Upright, similar to Black-grass.
- Fat Hen: Erect with a bushy appearance.
- Loose Silky-bent: Fine, tufted growth.
- Maize: Tall and upright with a single main stem.
- Scentless Mayweed: Erect and branching.
- Shepherds Purse: Basal rosette with a flowering stem.
- Small-flowered Cranesbill: Low-growing and spreading.
- Sugar beet: Rosette of leaves with a large taproot.
Challenges:
Some seedlings may have similar features, especially at early growth stages, making classification challenging. Variations in lighting, image quality, and seedling age can further complicate the task.
Checking for data imbalance¶
#Plot the countplot of each category
sns.countplot(labels['Label'])
plt.xticks(rotation='vertical')
(array([ 0., 100., 200., 300., 400., 500., 600., 700.]), [Text(0.0, 0, '0'), Text(100.0, 0, '100'), Text(200.0, 0, '200'), Text(300.0, 0, '300'), Text(400.0, 0, '400'), Text(500.0, 0, '500'), Text(600.0, 0, '600'), Text(700.0, 0, '700')])
- As you can see from the above plot, the dataset is quite imbalanced.
#count each plant species
labels.value_counts()
| 0 | |
|---|---|
| Label | |
| Loose Silky-bent | 654 |
| Common Chickweed | 611 |
| Scentless Mayweed | 516 |
| Small-flowered Cranesbill | 496 |
| Fat Hen | 475 |
| Charlock | 390 |
| Sugar beet | 385 |
| Cleavers | 287 |
| Black-grass | 263 |
| Shepherds Purse | 231 |
| Maize | 221 |
| Common wheat | 221 |
Observations from EDA
- dataset is quite imbalanced.
- Range of each categories varies from 221 to 654.
- Loose Silky-bent is the highest category and has around 654 images
- Second highest category is Common Chickweed around 611 images
- Maize and Common wheat have around 221 images lowest count in the dataset
Data Pre-Processing¶
Convert the BGR images to RGB images.¶
#Check before RGB conversion in matplotlib
plt.imshow(images[8])
plt.title(labels.iloc[8,0])
plt.show()
# Converting the images from BGR to RGB using cvtColor function of OpenCV
for i in range(len(images)):
images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)
#Check after RGB conversion in matplotlib each category
plot_images(images,labels)
#Display one image after RGB conversion in matplotlib
plt.imshow(images[8])
plt.title(labels.iloc[8,0])
plt.show()
Resize the images¶
As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.
#reduce the size of the image
resized_images = [] # Create a new list to store resized images
for i in range(len(images)):
resized_image = cv2.resize(images[i], (64,64), interpolation = cv2.INTER_AREA)
resized_images.append(resized_image)
images = np.array(resized_images)
#Display images after resized images
plot_images(images,labels)
#Display one image with dimension info
plt.imshow(images[8])
<matplotlib.image.AxesImage at 0x7ff871ea8090>
-We observe reduced image Size 64 x 64 as compared to original 124 X 124
Data Preparation for Modeling¶
- Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
- You'll have to encode categorical features and scale the pixel values.
- You will build a model using the train data and then check its performance
Split the dataset¶
- We will use 20% of our data for testing, 20% of our data for validation and 80% of our data for training.
- We are using the train_test_split() function from scikit-learn. Here, we split the dataset into three parts, train,test and validation.
images.shape
(4750, 64, 64, 3)
X_temp ,X_test, y_temp,y_test = train_test_split(images, labels, test_size=0.2, random_state=42,stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.2, random_state=42,stratify=y_temp)
print('Shape of X_train:', X_train.shape)
print('Shape of X_val:', X_val.shape)
print('Shape of X_test:', X_test.shape)
Shape of X_train: (3040, 64, 64, 3) Shape of X_val: (760, 64, 64, 3) Shape of X_test: (950, 64, 64, 3)
print('Shape of y_train:', y_train.shape)
print('Shape of y_val:', y_val.shape)
print('Shape of y_test:', y_test.shape)
Shape of y_train: (3040, 1) Shape of y_val: (760, 1) Shape of y_test: (950, 1)
# Check count of each class training dataset
unique_categories, category_counts = np.unique(y_train, return_counts=True)
# Print the category names and their counts
for category, count in zip(unique_categories, category_counts):
print(f"{category}: {count}")
Black-grass: 168 Charlock: 250 Cleavers: 183 Common Chickweed: 391 Common wheat: 142 Fat Hen: 304 Loose Silky-bent: 418 Maize: 142 Scentless Mayweed: 330 Shepherds Purse: 148 Small-flowered Cranesbill: 318 Sugar beet: 246
Encode the target labels¶
# Convert labels from names to one hot vectors.
# Labelbinarizer works similar to onehotencoder
from sklearn.preprocessing import LabelBinarizer
enc = LabelBinarizer()
# Applying fit_transform on train target variable
y_train_encoded = enc.fit_transform(y_train)
# Applying only transform on val and test target variable
y_val_encoded=enc.transform(y_val)
y_test_encoded=enc.transform(y_test)
Data Normalization¶
Since the image pixel values range from 0-255, our method of normalization here will be scaling - we shall divide all the pixel values by 255 to standardize the images to have values between 0-1.
# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0
Model Building¶
Let's create a CNN model sequentially, where we will be adding the layers one after another.
Helper Functions for Model Building¶
def plot(history, name):
"""
Function to plot loss/accuracy
history: an object which stores the metrics and losses.
name: can be one of Loss or Accuracy
"""
fig, ax = plt.subplots() #Creating a subplot with figure and axes.
plt.plot(history.history[name]) #Plotting the train accuracy or train loss
plt.plot(history.history['val_'+name]) #Plotting the validation accuracy or validation loss
plt.title('Model ' + name.capitalize()) #Defining the title of the plot.
plt.ylabel(name.capitalize()) #Capitalizing the first letter.
plt.xlabel('Epoch') #Defining the label for the x-axis.
fig.legend(['Train', 'Validation'], loc="outside right upper") #Defining the legend, loc controls the position of the legend.
# defining a function to compute different metrics to check performance of a classification model built using statsmodels
def model_performance_classification(
model, predictors, target, threshold=0.5
):
"""
Function to compute different metrics to check classification model performance
model: classifier
predictors: independent variables
target: dependent variable
threshold: threshold for classifying the observation as class 1
The choice of the threshold depends on the specific business problem and the desired balance between different types of errors:
Higher Threshold (e.g., 0.7): This leads to higher precision (fewer false positives) but lower recall (more false negatives).
It's suitable when the cost of false positives is high, like in spam detection.
Lower Threshold (e.g., 0.3): This leads to higher recall (fewer false negatives) but lower precision (more false positives).
It's suitable when the cost of false negatives is high, like in disease diagnosis.
Using threshold of 0.5 in this case
"""
# checking which probabilities are greater than threshold
pred = model.predict(predictors) > threshold
# pred_temp = model.predict(predictors) > threshold
# # rounding off the above values to get classes
# pred = np.round(pred_temp)
acc = accuracy_score(target, pred) # to compute Accuracy
recall = recall_score(target, pred,average='weighted',zero_division=1) # to compute Recall
precision = precision_score(target, pred,average='weighted',zero_division=1) # to compute Precision
f1 = f1_score(target, pred,average='weighted',zero_division=1) # to compute F1-score
# creating a dataframe of metrics
df_perf = pd.DataFrame(
{"Accuracy": acc, "Recall": recall, "Precision": precision, "F1 Score": f1,},
index=[0],
)
return df_perf
# Plots the heat map if tensorflow confusion matrix
def plot_confusion_matrix(y_test_classes,y_pred_classes):
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_classes,y_pred_classes)
f, ax = plt.subplots(figsize=(10, 8))
plt.title('Confusion Matrix')
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.ylabel('Actual label')
plt.xlabel('Predicted label')
plt.show()
Model Evaluation Criterion¶
F1-Score for Plant Seedling Classification. We will look at all the metrics to review the model performance. Accuracy is used to compile the model and also used to check the overall performance along with F1-score.
Potential Class Imbalance: Plant seedling datasets has class imbalance, where some species are more represented than others. Accuracy can be misleading in such cases.
Importance of Correct Identification: In agriculture, correctly identifying all instances of a particular species (especially weeds or invasive plants) might be crucial for timely intervention. This aligns with the focus of recall and the F1-score.
Comprehensive Evaluation: The F1-score provides a more balanced view of the model's performance, considering both precision and recall. It helps ensure the model is not only accurate overall but also effective in identifying all instances of important classes
Model1 : CNN Model¶
First, we need to clear the previous model's history from the session even if a single model can run multiple times on the same data.
In Keras, we need a special command to clear the model's history, otherwise the previous model history remains in the backend.
Also, let's fix the seed again after clearing the backend.
Let's set the seed for random number generators in Numpy, the Random library in Python, and in TensorFlow to be able to reproduce the same results every time we run the code.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
Now, let's build a CNN Model with the following 2 main parts -
- The Feature Extraction layers which are comprised of convolutional and pooling layers.
- The Fully Connected classification layers for prediction.
# Intializing a sequential model
model1 = Sequential()
#Define the convolutional layer for Feature Extraction
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model1.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2)))
model1.add(Conv2D(32, (3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())
# Adding a fully connected dense layer
model1.add(Dense(16, activation='relu'))
model1.add(Dropout(0.3))
model1.add(Dense(12, activation='softmax'))
# Adam as Optimizer
opt=Adam()
# Compile model
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model1.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 62, 62, 64) 1792
max_pooling2d (MaxPooling2 (None, 31, 31, 64) 0
D)
conv2d_1 (Conv2D) (None, 29, 29, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 14, 14, 32) 0
g2D)
flatten (Flatten) (None, 6272) 0
dense (Dense) (None, 16) 100368
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 120828 (471.98 KB)
Trainable params: 120828 (471.98 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on training data¶
start = time.time()
history_1 = model1.fit(
X_train_normalized, y_train_encoded,
epochs=30,
validation_data=(X_val_normalized,y_val_encoded),
batch_size=32,
verbose=2
)
end=time.time()
Epoch 1/30 95/95 - 6s - loss: 2.4337 - accuracy: 0.1395 - val_loss: 2.3158 - val_accuracy: 0.2132 - 6s/epoch - 63ms/step Epoch 2/30 95/95 - 5s - loss: 2.2039 - accuracy: 0.2421 - val_loss: 1.9942 - val_accuracy: 0.3421 - 5s/epoch - 53ms/step Epoch 3/30 95/95 - 5s - loss: 2.0104 - accuracy: 0.2964 - val_loss: 1.8476 - val_accuracy: 0.3566 - 5s/epoch - 57ms/step Epoch 4/30 95/95 - 5s - loss: 1.9094 - accuracy: 0.2947 - val_loss: 1.7443 - val_accuracy: 0.4053 - 5s/epoch - 53ms/step Epoch 5/30 95/95 - 5s - loss: 1.8613 - accuracy: 0.3401 - val_loss: 1.6699 - val_accuracy: 0.3987 - 5s/epoch - 54ms/step Epoch 6/30 95/95 - 5s - loss: 1.7871 - accuracy: 0.3605 - val_loss: 1.6050 - val_accuracy: 0.4118 - 5s/epoch - 54ms/step Epoch 7/30 95/95 - 5s - loss: 1.7667 - accuracy: 0.3592 - val_loss: 1.6642 - val_accuracy: 0.4526 - 5s/epoch - 51ms/step Epoch 8/30 95/95 - 5s - loss: 1.7447 - accuracy: 0.3503 - val_loss: 1.5809 - val_accuracy: 0.4461 - 5s/epoch - 55ms/step Epoch 9/30 95/95 - 5s - loss: 1.7242 - accuracy: 0.3549 - val_loss: 1.5149 - val_accuracy: 0.4724 - 5s/epoch - 52ms/step Epoch 10/30 95/95 - 5s - loss: 1.7157 - accuracy: 0.3694 - val_loss: 1.5305 - val_accuracy: 0.4855 - 5s/epoch - 50ms/step Epoch 11/30 95/95 - 5s - loss: 1.6712 - accuracy: 0.3724 - val_loss: 1.5024 - val_accuracy: 0.4421 - 5s/epoch - 56ms/step Epoch 12/30 95/95 - 5s - loss: 1.6593 - accuracy: 0.3829 - val_loss: 1.5065 - val_accuracy: 0.4842 - 5s/epoch - 53ms/step Epoch 13/30 95/95 - 5s - loss: 1.6503 - accuracy: 0.3845 - val_loss: 1.4744 - val_accuracy: 0.4711 - 5s/epoch - 54ms/step Epoch 14/30 95/95 - 5s - loss: 1.6549 - accuracy: 0.3852 - val_loss: 1.4746 - val_accuracy: 0.4776 - 5s/epoch - 52ms/step Epoch 15/30 95/95 - 5s - loss: 1.6231 - accuracy: 0.3885 - val_loss: 1.4525 - val_accuracy: 0.4934 - 5s/epoch - 51ms/step Epoch 16/30 95/95 - 5s - loss: 1.6115 - accuracy: 0.4000 - val_loss: 1.4287 - val_accuracy: 0.4632 - 5s/epoch - 54ms/step Epoch 17/30 95/95 - 5s - loss: 1.5881 - accuracy: 0.4092 - val_loss: 1.4068 - val_accuracy: 0.5053 - 5s/epoch - 51ms/step Epoch 18/30 95/95 - 5s - loss: 1.5890 - accuracy: 0.4109 - val_loss: 1.4431 - val_accuracy: 0.4855 - 5s/epoch - 52ms/step Epoch 19/30 95/95 - 5s - loss: 1.5707 - accuracy: 0.4118 - val_loss: 1.4149 - val_accuracy: 0.5158 - 5s/epoch - 56ms/step Epoch 20/30 95/95 - 5s - loss: 1.5632 - accuracy: 0.4171 - val_loss: 1.3900 - val_accuracy: 0.5224 - 5s/epoch - 51ms/step Epoch 21/30 95/95 - 5s - loss: 1.5439 - accuracy: 0.4178 - val_loss: 1.3526 - val_accuracy: 0.5539 - 5s/epoch - 52ms/step Epoch 22/30 95/95 - 5s - loss: 1.5589 - accuracy: 0.4204 - val_loss: 1.3067 - val_accuracy: 0.5803 - 5s/epoch - 52ms/step Epoch 23/30 95/95 - 5s - loss: 1.5052 - accuracy: 0.4336 - val_loss: 1.3088 - val_accuracy: 0.5737 - 5s/epoch - 51ms/step Epoch 24/30 95/95 - 5s - loss: 1.4899 - accuracy: 0.4276 - val_loss: 1.3369 - val_accuracy: 0.5724 - 5s/epoch - 54ms/step Epoch 25/30 95/95 - 5s - loss: 1.4565 - accuracy: 0.4457 - val_loss: 1.2902 - val_accuracy: 0.5737 - 5s/epoch - 51ms/step Epoch 26/30 95/95 - 5s - loss: 1.4872 - accuracy: 0.4411 - val_loss: 1.4357 - val_accuracy: 0.5039 - 5s/epoch - 51ms/step Epoch 27/30 95/95 - 5s - loss: 1.4517 - accuracy: 0.4543 - val_loss: 1.2848 - val_accuracy: 0.5671 - 5s/epoch - 54ms/step Epoch 28/30 95/95 - 5s - loss: 1.4411 - accuracy: 0.4523 - val_loss: 1.2773 - val_accuracy: 0.6105 - 5s/epoch - 52ms/step Epoch 29/30 95/95 - 5s - loss: 1.4297 - accuracy: 0.4674 - val_loss: 1.2595 - val_accuracy: 0.5803 - 5s/epoch - 52ms/step Epoch 30/30 95/95 - 5s - loss: 1.4036 - accuracy: 0.4697 - val_loss: 1.2424 - val_accuracy: 0.5763 - 5s/epoch - 52ms/step
model1_time = end-start
print("Time taken in minutes ",model1_time/60)
Time taken in minutes 2.528558937708537
Model Evaluation¶
#Plot of Model Loss
plot(history_1,'loss')
#plot of model accuracy
plot(history_1,'accuracy')
#Evaluate the model on test data
#model.evaluate() is used to evaluate the model's performance on the test dataset (X_test_normalized, y_test_encoded). This is the correct approach to get a final, unbiased assessment of the trained model
accuracy1 = model1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
30/30 - 0s - loss: 1.2004 - accuracy: 0.6084 - 332ms/epoch - 11ms/step
#Predictions on test data
# Here we would get the output as probablities for each category
y_pred=model1.predict(X_test_normalized)
30/30 [==============================] - 0s 12ms/step
Model Performance and Confusion Matrix¶
# Model Performance on training data
model1_train_perf = model_performance_classification(model1,X_train_normalized,y_train_encoded)
print(model1_train_perf)
95/95 [==============================] - 1s 13ms/step Accuracy Recall Precision F1 Score 0 0.437829 0.437829 0.889654 0.4764
# Model Performance on validation data
model1_val_perf = model_performance_classification(model1,X_val_normalized,y_val_encoded)
print(model1_val_perf)
24/24 [==============================] - 0s 14ms/step Accuracy Recall Precision F1 Score 0 0.351316 0.351316 0.662824 0.39743
# Model Performance on testing data
model1_test_perf = model_performance_classification(model1,X_test_normalized,y_test_encoded)
print(model1_test_perf)
30/30 [==============================] - 0s 14ms/step Accuracy Recall Precision F1 Score 0 0.372632 0.372632 0.739194 0.401501
We will use the argmax() function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes
# Obtaining the categorical values from y_test_encoded and y_pred
y_test_classes = np.argmax(y_test_encoded, axis=1)
y_pred_classes = np.argmax(y_pred, axis=1)
# Performance summary using Classification report
report1 = classification_report(y_test_classes, y_pred_classes,zero_division=1)
print(report1)
precision recall f1-score support
0 1.00 0.00 0.00 53
1 0.89 0.74 0.81 78
2 0.74 0.34 0.47 58
3 0.62 0.85 0.71 122
4 0.00 0.00 0.00 44
5 0.50 0.64 0.56 95
6 0.54 0.95 0.69 131
7 1.00 0.00 0.00 44
8 0.58 0.70 0.63 103
9 0.36 0.09 0.14 46
10 0.78 0.84 0.81 99
11 0.57 0.66 0.61 77
accuracy 0.61 950
macro avg 0.63 0.49 0.45 950
weighted avg 0.63 0.61 0.55 950
The classification report would help you determine how well the CNN model is identifying different plant species. By analyzing the metrics for each species, we can identify potential weaknesses (e.g., species that are frequently misclassified) and guide further model improvements.
Test Data Performance
- We observe accuracy of 61%
- Macro Average F1 score of 45%
- Weighted average F1 score of 55%
#call the defined helper function to display the heatmap of confusion_matrix
plot_confusion_matrix(y_test_classes,y_pred_classes)
Observations:TensorFlow's confusion matrix function provides the raw counts
- We observe that majority of the classes are not predicted correctly.Accuracy of 61% on test data.
- In comparison to the rest, we can see that classes 1, 3 ,5,6 , 8,10 and 11 are classified better.
- We can also observe that classes 0,7 and 4 are mostly misclassified.
Model Performance Improvement¶
As we can see, our initial model appears to overfit. Therefore we'll try to address this problem with data augmentation to check if we can improve the model's performance.
Reducing the Learning Rate:
Hint: Use ReduceLRonPlateau() function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.
Model 2: Data Augmentation Model - Rotation Technique¶
Images are rotated
Remember, data augmentation should not be used in the validation/test data set.
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Data generator with rotation set
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
# Intializing a sequential model
model2 = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model2.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2)))
model2.add(Conv2D(32, (3, 3), activation='relu'))
model2.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())
model2.add(Dense(16, activation='relu'))
model2.add(Dropout(0.3))
model2.add(Dense(12, activation='softmax'))
#Use Adam Optimizer
opt=Adam()
# Compile model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model2.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 62, 62, 64) 1792
max_pooling2d (MaxPooling2 (None, 31, 31, 64) 0
D)
conv2d_1 (Conv2D) (None, 29, 29, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 14, 14, 32) 0
g2D)
flatten (Flatten) (None, 6272) 0
dense (Dense) (None, 16) 100368
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 120828 (471.98 KB)
Trainable params: 120828 (471.98 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on training data¶
# Epochs
epochs = 30
# Batch size
batch_size = 64
augmented_data = train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
seed=42,
shuffle=False)
start = time.time()
history_2 = model2.fit(augmented_data,
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=2)
end = time.time()
Epoch 1/30 47/47 - 7s - loss: 2.4433 - accuracy: 0.1376 - val_loss: 2.4023 - val_accuracy: 0.1132 - 7s/epoch - 146ms/step Epoch 2/30 47/47 - 6s - loss: 2.3785 - accuracy: 0.1821 - val_loss: 2.2719 - val_accuracy: 0.2526 - 6s/epoch - 130ms/step Epoch 3/30 47/47 - 6s - loss: 2.2037 - accuracy: 0.2248 - val_loss: 2.0605 - val_accuracy: 0.2697 - 6s/epoch - 123ms/step Epoch 4/30 47/47 - 6s - loss: 2.0335 - accuracy: 0.2718 - val_loss: 1.8806 - val_accuracy: 0.4487 - 6s/epoch - 126ms/step Epoch 5/30 47/47 - 6s - loss: 1.9312 - accuracy: 0.3189 - val_loss: 1.7497 - val_accuracy: 0.4684 - 6s/epoch - 126ms/step Epoch 6/30 47/47 - 6s - loss: 1.8355 - accuracy: 0.3666 - val_loss: 1.6087 - val_accuracy: 0.4908 - 6s/epoch - 125ms/step Epoch 7/30 47/47 - 6s - loss: 1.7458 - accuracy: 0.3770 - val_loss: 1.5568 - val_accuracy: 0.5224 - 6s/epoch - 123ms/step Epoch 8/30 47/47 - 6s - loss: 1.6789 - accuracy: 0.4049 - val_loss: 1.4661 - val_accuracy: 0.5197 - 6s/epoch - 130ms/step Epoch 9/30 47/47 - 6s - loss: 1.6594 - accuracy: 0.3915 - val_loss: 1.3899 - val_accuracy: 0.5197 - 6s/epoch - 122ms/step Epoch 10/30 47/47 - 6s - loss: 1.6252 - accuracy: 0.4083 - val_loss: 1.4251 - val_accuracy: 0.5303 - 6s/epoch - 128ms/step Epoch 11/30 47/47 - 6s - loss: 1.6046 - accuracy: 0.4251 - val_loss: 1.3470 - val_accuracy: 0.5211 - 6s/epoch - 123ms/step Epoch 12/30 47/47 - 6s - loss: 1.5779 - accuracy: 0.4278 - val_loss: 1.3567 - val_accuracy: 0.5368 - 6s/epoch - 127ms/step Epoch 13/30 47/47 - 6s - loss: 1.5639 - accuracy: 0.4261 - val_loss: 1.3444 - val_accuracy: 0.5395 - 6s/epoch - 126ms/step Epoch 14/30 47/47 - 6s - loss: 1.5497 - accuracy: 0.4409 - val_loss: 1.3333 - val_accuracy: 0.5289 - 6s/epoch - 125ms/step Epoch 15/30 47/47 - 6s - loss: 1.5161 - accuracy: 0.4479 - val_loss: 1.3011 - val_accuracy: 0.5368 - 6s/epoch - 129ms/step Epoch 16/30 47/47 - 6s - loss: 1.5067 - accuracy: 0.4412 - val_loss: 1.2479 - val_accuracy: 0.5645 - 6s/epoch - 124ms/step Epoch 17/30 47/47 - 6s - loss: 1.4926 - accuracy: 0.4516 - val_loss: 1.2757 - val_accuracy: 0.5592 - 6s/epoch - 130ms/step Epoch 18/30 47/47 - 6s - loss: 1.4921 - accuracy: 0.4395 - val_loss: 1.2338 - val_accuracy: 0.5526 - 6s/epoch - 125ms/step Epoch 19/30 47/47 - 6s - loss: 1.4515 - accuracy: 0.4637 - val_loss: 1.2393 - val_accuracy: 0.5605 - 6s/epoch - 130ms/step Epoch 20/30 47/47 - 6s - loss: 1.4776 - accuracy: 0.4637 - val_loss: 1.2529 - val_accuracy: 0.5671 - 6s/epoch - 125ms/step Epoch 21/30 47/47 - 6s - loss: 1.4879 - accuracy: 0.4546 - val_loss: 1.3796 - val_accuracy: 0.5592 - 6s/epoch - 126ms/step Epoch 22/30 47/47 - 6s - loss: 1.4687 - accuracy: 0.4614 - val_loss: 1.2633 - val_accuracy: 0.5882 - 6s/epoch - 126ms/step Epoch 23/30 47/47 - 6s - loss: 1.4249 - accuracy: 0.4718 - val_loss: 1.2267 - val_accuracy: 0.5829 - 6s/epoch - 124ms/step Epoch 24/30 47/47 - 6s - loss: 1.4560 - accuracy: 0.4688 - val_loss: 1.1907 - val_accuracy: 0.6079 - 6s/epoch - 128ms/step Epoch 25/30 47/47 - 6s - loss: 1.4095 - accuracy: 0.4735 - val_loss: 1.1517 - val_accuracy: 0.6092 - 6s/epoch - 121ms/step Epoch 26/30 47/47 - 6s - loss: 1.4035 - accuracy: 0.4745 - val_loss: 1.1801 - val_accuracy: 0.5934 - 6s/epoch - 128ms/step Epoch 27/30 47/47 - 6s - loss: 1.4007 - accuracy: 0.4778 - val_loss: 1.1652 - val_accuracy: 0.5842 - 6s/epoch - 121ms/step Epoch 28/30 47/47 - 6s - loss: 1.3830 - accuracy: 0.4812 - val_loss: 1.1555 - val_accuracy: 0.5868 - 6s/epoch - 127ms/step Epoch 29/30 47/47 - 6s - loss: 1.4200 - accuracy: 0.4758 - val_loss: 1.2066 - val_accuracy: 0.5816 - 6s/epoch - 125ms/step Epoch 30/30 47/47 - 6s - loss: 1.4470 - accuracy: 0.4556 - val_loss: 1.1466 - val_accuracy: 0.5842 - 6s/epoch - 126ms/step
model2_time = end-start
print("Time taken in minutes ",model2_time/60)
Time taken in minutes 3.0522127310434977
Model Evaluation¶
#Plot of Model Loss
plot(history_2,'loss')
#plot of model accuracy
plot(history_2,'accuracy')
#Evaluate the model on test data
accuracy2 = model2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
30/30 - 0s - loss: 1.1469 - accuracy: 0.5863 - 391ms/epoch - 13ms/step
#Predictions on test data
# Here we would get the output as probablities for each category
y_pred=model2.predict(X_test_normalized)
30/30 [==============================] - 0s 12ms/step
Model Performance and Confusion Matrix¶
# Model Performance on training data
model2_train_perf = model_performance_classification(model2,X_train_normalized,y_train_encoded)
print(model1_train_perf)
95/95 [==============================] - 1s 11ms/step Accuracy Recall Precision F1 Score 0 0.437829 0.437829 0.889654 0.4764
# Model Performance on validation data
model2_val_perf = model_performance_classification(model2,X_val_normalized,y_val_encoded)
print(model1_val_perf)
24/24 [==============================] - 0s 11ms/step Accuracy Recall Precision F1 Score 0 0.351316 0.351316 0.662824 0.39743
# Model Performance on testing data
model2_test_perf = model_performance_classification(model2,X_test_normalized,y_test_encoded)
print(model2_test_perf)
30/30 [==============================] - 0s 11ms/step Accuracy Recall Precision F1 Score 0 0.397895 0.397895 0.809316 0.422596
We will use the argmax() function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes
# Obtaining the categorical values from y_test_encoded and y_pred
y_test_classes = np.argmax(y_test_encoded, axis=1)
y_pred_classes = np.argmax(y_pred, axis=1)
# Performance summary using Classification report
report2 = classification_report(y_test_classes, y_pred_classes,zero_division=1)
print(report2)
precision recall f1-score support
0 1.00 0.00 0.00 53
1 0.82 0.78 0.80 78
2 0.74 0.69 0.71 58
3 0.60 0.93 0.73 122
4 1.00 0.00 0.00 44
5 0.40 0.55 0.46 95
6 0.52 0.98 0.68 131
7 1.00 0.00 0.00 44
8 0.59 0.69 0.63 103
9 0.31 0.11 0.16 46
10 0.76 0.74 0.75 99
11 0.57 0.17 0.26 77
accuracy 0.59 950
macro avg 0.69 0.47 0.43 950
weighted avg 0.65 0.59 0.52 950
The classification report would help you determine how well the CNN model is identifying different plant species. By analyzing the metrics for each species, we can identify potential weaknesses (e.g., species that are frequently misclassified) and guide further model improvements.
- We observe accuracy of 59%
- Macro Average F1 score of 43%
- Weighted average F1 score of 52%
#call the defined helper function to display the heatmap of confusion_matrix
plot_confusion_matrix(y_test_classes,y_pred_classes)
Observations:TensorFlow's confusion matrix function provides the raw counts
- We observe that majority of the classes are not predicted correctly. Model is performing with accuracy of 59%.
- In comparison to the rest, we can see that classes 3, 6 , 10 are classified better.
- 0 ,4,7 are mostly wrongly predicted.
- We can also observe that classes 9 and 11 are poorly misclassified.
Model 3: Data Augmentation Model with other augmentation techniques¶
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
# Augmentation techniques used.
train_datagen = ImageDataGenerator(
horizontal_flip = True,
vertical_flip = False,
height_shift_range= 0.1,
width_shift_range=0.1,
rotation_range=20,
shear_range = 0.1,
zoom_range=0.1,
fill_mode='nearest'
)
# Intializing a sequential model
model3 = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model3.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model3.add(MaxPooling2D((2, 2)))
model3.add(Conv2D(32, (3, 3), activation='relu'))
model3.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model3.add(Flatten())
model3.add(Dense(16, activation='relu'))
model3.add(Dropout(0.3))
model3.add(Dense(12, activation='softmax'))
#Use Adam Optimizer
opt=Adam()
# Compile model
model3.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model3.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 62, 62, 64) 1792
max_pooling2d (MaxPooling2 (None, 31, 31, 64) 0
D)
conv2d_1 (Conv2D) (None, 29, 29, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 14, 14, 32) 0
g2D)
flatten (Flatten) (None, 6272) 0
dense (Dense) (None, 16) 100368
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 120828 (471.98 KB)
Trainable params: 120828 (471.98 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on training data¶
# Epochs
epochs = 30
# Batch size
batch_size = 64
augmented_data = train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
seed=42,
shuffle=False)
start = time.time()
history_3 = model3.fit(augmented_data,
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=2)
end = time.time()
Epoch 1/30 47/47 - 8s - loss: 2.3958 - accuracy: 0.1657 - val_loss: 2.2585 - val_accuracy: 0.2211 - 8s/epoch - 161ms/step Epoch 2/30 47/47 - 7s - loss: 2.2400 - accuracy: 0.2483 - val_loss: 2.1347 - val_accuracy: 0.3000 - 7s/epoch - 139ms/step Epoch 3/30 47/47 - 6s - loss: 2.1532 - accuracy: 0.2735 - val_loss: 1.9928 - val_accuracy: 0.3461 - 6s/epoch - 136ms/step Epoch 4/30 47/47 - 6s - loss: 2.0896 - accuracy: 0.2981 - val_loss: 1.9293 - val_accuracy: 0.3737 - 6s/epoch - 135ms/step Epoch 5/30 47/47 - 6s - loss: 2.0361 - accuracy: 0.3112 - val_loss: 1.8478 - val_accuracy: 0.3868 - 6s/epoch - 136ms/step Epoch 6/30 47/47 - 6s - loss: 1.9676 - accuracy: 0.3118 - val_loss: 1.7258 - val_accuracy: 0.4026 - 6s/epoch - 136ms/step Epoch 7/30 47/47 - 6s - loss: 1.9219 - accuracy: 0.3276 - val_loss: 1.7134 - val_accuracy: 0.4316 - 6s/epoch - 135ms/step Epoch 8/30 47/47 - 6s - loss: 1.9023 - accuracy: 0.3303 - val_loss: 1.7075 - val_accuracy: 0.4539 - 6s/epoch - 131ms/step Epoch 9/30 47/47 - 6s - loss: 1.8816 - accuracy: 0.3286 - val_loss: 1.6844 - val_accuracy: 0.4408 - 6s/epoch - 135ms/step Epoch 10/30 47/47 - 6s - loss: 1.8455 - accuracy: 0.3296 - val_loss: 1.6138 - val_accuracy: 0.4487 - 6s/epoch - 132ms/step Epoch 11/30 47/47 - 6s - loss: 1.8209 - accuracy: 0.3390 - val_loss: 1.5973 - val_accuracy: 0.4921 - 6s/epoch - 135ms/step Epoch 12/30 47/47 - 6s - loss: 1.8283 - accuracy: 0.3273 - val_loss: 1.5518 - val_accuracy: 0.4803 - 6s/epoch - 130ms/step Epoch 13/30 47/47 - 6s - loss: 1.7694 - accuracy: 0.3528 - val_loss: 1.5077 - val_accuracy: 0.5197 - 6s/epoch - 134ms/step Epoch 14/30 47/47 - 6s - loss: 1.7733 - accuracy: 0.3612 - val_loss: 1.6173 - val_accuracy: 0.4776 - 6s/epoch - 129ms/step Epoch 15/30 47/47 - 6s - loss: 1.7703 - accuracy: 0.3478 - val_loss: 1.4884 - val_accuracy: 0.5184 - 6s/epoch - 136ms/step Epoch 16/30 47/47 - 6s - loss: 1.7564 - accuracy: 0.3673 - val_loss: 1.4911 - val_accuracy: 0.4961 - 6s/epoch - 130ms/step Epoch 17/30 47/47 - 6s - loss: 1.7520 - accuracy: 0.3656 - val_loss: 1.4541 - val_accuracy: 0.5171 - 6s/epoch - 134ms/step Epoch 18/30 47/47 - 6s - loss: 1.7309 - accuracy: 0.3720 - val_loss: 1.4875 - val_accuracy: 0.5250 - 6s/epoch - 131ms/step Epoch 19/30 47/47 - 6s - loss: 1.7044 - accuracy: 0.3753 - val_loss: 1.4843 - val_accuracy: 0.5382 - 6s/epoch - 132ms/step Epoch 20/30 47/47 - 6s - loss: 1.7523 - accuracy: 0.3642 - val_loss: 1.5248 - val_accuracy: 0.5145 - 6s/epoch - 133ms/step Epoch 21/30 47/47 - 6s - loss: 1.7386 - accuracy: 0.3750 - val_loss: 1.3878 - val_accuracy: 0.5908 - 6s/epoch - 131ms/step Epoch 22/30 47/47 - 6s - loss: 1.6980 - accuracy: 0.3827 - val_loss: 1.4117 - val_accuracy: 0.5500 - 6s/epoch - 132ms/step Epoch 23/30 47/47 - 6s - loss: 1.7096 - accuracy: 0.3824 - val_loss: 1.3797 - val_accuracy: 0.5671 - 6s/epoch - 129ms/step Epoch 24/30 47/47 - 6s - loss: 1.6628 - accuracy: 0.3945 - val_loss: 1.3512 - val_accuracy: 0.5803 - 6s/epoch - 134ms/step Epoch 25/30 47/47 - 6s - loss: 1.6364 - accuracy: 0.3992 - val_loss: 1.3705 - val_accuracy: 0.5658 - 6s/epoch - 129ms/step Epoch 26/30 47/47 - 6s - loss: 1.6273 - accuracy: 0.4133 - val_loss: 1.3373 - val_accuracy: 0.5737 - 6s/epoch - 132ms/step Epoch 27/30 47/47 - 6s - loss: 1.6829 - accuracy: 0.3854 - val_loss: 1.3582 - val_accuracy: 0.5816 - 6s/epoch - 132ms/step Epoch 28/30 47/47 - 6s - loss: 1.6250 - accuracy: 0.3915 - val_loss: 1.3169 - val_accuracy: 0.5789 - 6s/epoch - 136ms/step Epoch 29/30 47/47 - 6s - loss: 1.6105 - accuracy: 0.4140 - val_loss: 1.2984 - val_accuracy: 0.5789 - 6s/epoch - 132ms/step Epoch 30/30 47/47 - 6s - loss: 1.6046 - accuracy: 0.4103 - val_loss: 1.3579 - val_accuracy: 0.5632 - 6s/epoch - 134ms/step
model3_time = end-start
print("Time taken in minutes ",model3_time/60)
Time taken in minutes 3.156686536471049
Model Evaluation¶
#Plot of Model Loss
plot(history_3,'loss')
#plot of model accuracy
plot(history_3,'accuracy')
#Evaluate the model on test data
accuracy3 = model3.evaluate(X_test_normalized, y_test_encoded, verbose=2)
30/30 - 0s - loss: 1.3592 - accuracy: 0.5726 - 369ms/epoch - 12ms/step
#Predictions on test data
# Here we would get the output as probablities for each category
y_pred=model3.predict(X_test_normalized)
30/30 [==============================] - 0s 13ms/step
Model Performance and Confusion Matrix¶
# Model Performance on training data
model3_train_perf = model_performance_classification(model3,X_train_normalized,y_train_encoded)
print(model3_train_perf)
95/95 [==============================] - 1s 12ms/step Accuracy Recall Precision F1 Score 0 0.217434 0.217434 0.833575 0.266767
# Model Performance on validation data
model3_val_perf = model_performance_classification(model3,X_val_normalized,y_val_encoded)
print(model3_val_perf)
24/24 [==============================] - 0s 11ms/step Accuracy Recall Precision F1 Score 0 0.206579 0.206579 0.949299 0.255691
# Model Performance on testing data
model3_test_perf = model_performance_classification(model3,X_test_normalized,y_test_encoded)
print(model3_test_perf)
30/30 [==============================] - 0s 11ms/step Accuracy Recall Precision F1 Score 0 0.223158 0.223158 0.891805 0.266118
We will use the argmax() function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes
# Obtaining the categorical values from y_test_encoded and y_pred
y_test_classes = np.argmax(y_test_encoded, axis=1)
y_pred_classes = np.argmax(y_pred, axis=1)
# Performance summary using Classification report
report3 = classification_report(y_test_classes, y_pred_classes,zero_division=1)
print(report3)
precision recall f1-score support
0 1.00 0.00 0.00 53
1 0.45 0.96 0.62 78
2 0.57 0.45 0.50 58
3 0.61 0.93 0.74 122
4 1.00 0.00 0.00 44
5 0.63 0.36 0.46 95
6 0.60 0.90 0.72 131
7 1.00 0.00 0.00 44
8 0.58 0.62 0.60 103
9 0.33 0.02 0.04 46
10 0.64 0.88 0.74 99
11 0.50 0.32 0.39 77
accuracy 0.57 950
macro avg 0.66 0.45 0.40 950
weighted avg 0.63 0.57 0.50 950
The classification report would help you determine how well the CNN model is identifying different plant species. By analyzing the metrics for each species, we can identify potential weaknesses (e.g., species that are frequently misclassified) and guide further model improvements.
- We observe accuracy of 57%
- Macro Average F1 score of 40%
- Weighted average F1 score of 50%
#call the defined helper function to display the heatmap of confusion_matrix
plot_confusion_matrix(y_test_classes,y_pred_classes)
Observations:TensorFlow's confusion matrix function provides the raw counts
- Model accuracy of 57%
- We observe that majority of the classes are not predicted correctly.
- In comparison to the rest, we can see that classes 1,3 ,6 and 10 are classified better.
- We can also observe that classes 0,4,7 are classified worst.
Model 4: Data Augmentation - Model using ReduceLRonPlateau()¶
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
# Intializing a sequential model
model4 = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model4.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model4.add(MaxPooling2D((2, 2)))
model4.add(Conv2D(32, (3, 3), activation='relu'))
model4.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model4.add(Flatten())
model4.add(Dense(16, activation='relu'))
model4.add(Dropout(0.3))
model4.add(Dense(12, activation='softmax'))
#Use Adam Optimizer
opt=Adam()
# Compile model
model4.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model4.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 62, 62, 64) 1792
max_pooling2d (MaxPooling2 (None, 31, 31, 64) 0
D)
conv2d_1 (Conv2D) (None, 29, 29, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 14, 14, 32) 0
g2D)
flatten (Flatten) (None, 6272) 0
dense (Dense) (None, 16) 100368
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 120828 (471.98 KB)
Trainable params: 120828 (471.98 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Fitting the model on training data using ReduceLRonPlateau()¶
callbacks
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.001)
start = time.time()
history_4 = model4.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
seed=42,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
callbacks=[reduce_lr],
verbose=2)
end = time.time()
Epoch 1/30 47/47 - 7s - loss: 2.4520 - accuracy: 0.1149 - val_loss: 2.4276 - val_accuracy: 0.1382 - lr: 0.0010 - 7s/epoch - 145ms/step Epoch 2/30 47/47 - 7s - loss: 2.4406 - accuracy: 0.1243 - val_loss: 2.4193 - val_accuracy: 0.1382 - lr: 0.0010 - 7s/epoch - 156ms/step Epoch 3/30 47/47 - 6s - loss: 2.4315 - accuracy: 0.1203 - val_loss: 2.4055 - val_accuracy: 0.1382 - lr: 0.0010 - 6s/epoch - 137ms/step Epoch 4/30 47/47 - 6s - loss: 2.4173 - accuracy: 0.1502 - val_loss: 2.4110 - val_accuracy: 0.1382 - lr: 0.0010 - 6s/epoch - 124ms/step Epoch 5/30 47/47 - 6s - loss: 2.4054 - accuracy: 0.1495 - val_loss: 2.3643 - val_accuracy: 0.2000 - lr: 0.0010 - 6s/epoch - 129ms/step Epoch 6/30 47/47 - 6s - loss: 2.3630 - accuracy: 0.1640 - val_loss: 2.3032 - val_accuracy: 0.1974 - lr: 0.0010 - 6s/epoch - 124ms/step Epoch 7/30 47/47 - 6s - loss: 2.3134 - accuracy: 0.1996 - val_loss: 2.2062 - val_accuracy: 0.2500 - lr: 0.0010 - 6s/epoch - 129ms/step Epoch 8/30 47/47 - 6s - loss: 2.2348 - accuracy: 0.2345 - val_loss: 2.1151 - val_accuracy: 0.2592 - lr: 0.0010 - 6s/epoch - 123ms/step Epoch 9/30 47/47 - 6s - loss: 2.1544 - accuracy: 0.2295 - val_loss: 2.0459 - val_accuracy: 0.2803 - lr: 0.0010 - 6s/epoch - 125ms/step Epoch 10/30 47/47 - 6s - loss: 2.1195 - accuracy: 0.2392 - val_loss: 1.9659 - val_accuracy: 0.2921 - lr: 0.0010 - 6s/epoch - 124ms/step Epoch 11/30 47/47 - 6s - loss: 2.0794 - accuracy: 0.2487 - val_loss: 1.9595 - val_accuracy: 0.3526 - lr: 0.0010 - 6s/epoch - 124ms/step Epoch 12/30 47/47 - 6s - loss: 2.0109 - accuracy: 0.2806 - val_loss: 1.9340 - val_accuracy: 0.3592 - lr: 0.0010 - 6s/epoch - 129ms/step Epoch 13/30 47/47 - 6s - loss: 1.9999 - accuracy: 0.2792 - val_loss: 1.8504 - val_accuracy: 0.3355 - lr: 0.0010 - 6s/epoch - 123ms/step Epoch 14/30 47/47 - 6s - loss: 1.9722 - accuracy: 0.2950 - val_loss: 1.8451 - val_accuracy: 0.3842 - lr: 0.0010 - 6s/epoch - 130ms/step Epoch 15/30 47/47 - 6s - loss: 1.9679 - accuracy: 0.3007 - val_loss: 1.8453 - val_accuracy: 0.3789 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 16/30 47/47 - 6s - loss: 1.9490 - accuracy: 0.2994 - val_loss: 1.7959 - val_accuracy: 0.3697 - lr: 0.0010 - 6s/epoch - 134ms/step Epoch 17/30 47/47 - 6s - loss: 1.9424 - accuracy: 0.3125 - val_loss: 1.8060 - val_accuracy: 0.3711 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 18/30 47/47 - 6s - loss: 1.9618 - accuracy: 0.2984 - val_loss: 1.8169 - val_accuracy: 0.3605 - lr: 0.0010 - 6s/epoch - 124ms/step Epoch 19/30 47/47 - 6s - loss: 1.9090 - accuracy: 0.3128 - val_loss: 1.7505 - val_accuracy: 0.3750 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 20/30 47/47 - 6s - loss: 1.9106 - accuracy: 0.3162 - val_loss: 1.7363 - val_accuracy: 0.3803 - lr: 0.0010 - 6s/epoch - 126ms/step Epoch 21/30 47/47 - 6s - loss: 1.9048 - accuracy: 0.3219 - val_loss: 1.7632 - val_accuracy: 0.3868 - lr: 0.0010 - 6s/epoch - 138ms/step Epoch 22/30 47/47 - 6s - loss: 1.8849 - accuracy: 0.3246 - val_loss: 1.7402 - val_accuracy: 0.3816 - lr: 0.0010 - 6s/epoch - 129ms/step Epoch 23/30 47/47 - 6s - loss: 1.8809 - accuracy: 0.3145 - val_loss: 1.6993 - val_accuracy: 0.3566 - lr: 0.0010 - 6s/epoch - 132ms/step Epoch 24/30 47/47 - 6s - loss: 1.8583 - accuracy: 0.3259 - val_loss: 1.7327 - val_accuracy: 0.3816 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 25/30 47/47 - 6s - loss: 1.8531 - accuracy: 0.3303 - val_loss: 1.7597 - val_accuracy: 0.3868 - lr: 0.0010 - 6s/epoch - 134ms/step Epoch 26/30 47/47 - 6s - loss: 1.8690 - accuracy: 0.3280 - val_loss: 1.7021 - val_accuracy: 0.3882 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 27/30 47/47 - 6s - loss: 1.8448 - accuracy: 0.3280 - val_loss: 1.6918 - val_accuracy: 0.3803 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 28/30 47/47 - 6s - loss: 1.8479 - accuracy: 0.3306 - val_loss: 1.6913 - val_accuracy: 0.4000 - lr: 0.0010 - 6s/epoch - 135ms/step Epoch 29/30 47/47 - 6s - loss: 1.8358 - accuracy: 0.3381 - val_loss: 1.6902 - val_accuracy: 0.4000 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 30/30 47/47 - 6s - loss: 1.8399 - accuracy: 0.3337 - val_loss: 1.6782 - val_accuracy: 0.4000 - lr: 0.0010 - 6s/epoch - 130ms/step
model4_time = end-start
print("Time taken in minutes ",model4_time/60)
Time taken in minutes 3.477506466706594
Model Evaluation¶
#Plot of Model Loss
plot(history_4,'loss')
#plot of model accuracy
plot(history_4,'accuracy')
#Evaluate the model on test data
accuracy4 = model4.evaluate(X_test_normalized, y_test_encoded, verbose=2)
30/30 - 0s - loss: 1.6591 - accuracy: 0.4105 - 412ms/epoch - 14ms/step
#Predictions on test data
# Here we would get the output as probablities for each category
y_pred=model4.predict(X_test_normalized)
30/30 [==============================] - 0s 12ms/step
Model Performance and Confusion Matrix¶
# Model Performance on training data
model4_train_perf = model_performance_classification(model4,X_train_normalized,y_train_encoded)
print(model4_train_perf)
95/95 [==============================] - 1s 12ms/step Accuracy Recall Precision F1 Score 0 0.132566 0.132566 0.915927 0.144398
# Model Performance on validation data
model4_val_perf = model_performance_classification(model4,X_val_normalized,y_val_encoded)
print(model4_val_perf)
24/24 [==============================] - 0s 11ms/step Accuracy Recall Precision F1 Score 0 0.119737 0.119737 0.925943 0.139
# Model Performance on testing data
model4_test_perf = model_performance_classification(model4,X_test_normalized,y_test_encoded)
print(model4_test_perf)
30/30 [==============================] - 0s 11ms/step Accuracy Recall Precision F1 Score 0 0.127368 0.127368 0.91757 0.139435
We will use the argmax() function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes
# Obtaining the categorical values from y_test_encoded and y_pred
y_test_classes = np.argmax(y_test_encoded, axis=1)
y_pred_classes = np.argmax(y_pred, axis=1)
# Performance summary using Classification report
report4 = classification_report(y_test_classes, y_pred_classes,zero_division=1)
print(report4)
precision recall f1-score support
0 1.00 0.00 0.00 53
1 1.00 0.00 0.00 78
2 1.00 0.00 0.00 58
3 0.39 0.91 0.55 122
4 1.00 0.00 0.00 44
5 0.37 0.52 0.43 95
6 0.54 0.92 0.68 131
7 1.00 0.00 0.00 44
8 0.32 0.17 0.23 103
9 1.00 0.00 0.00 46
10 0.36 0.92 0.52 99
11 1.00 0.00 0.00 77
accuracy 0.41 950
macro avg 0.75 0.29 0.20 950
weighted avg 0.65 0.41 0.29 950
The classification report would help you determine how well the CNN model is identifying different plant species. By analyzing the metrics for each species, we can identify potential weaknesses (e.g., species that are frequently misclassified) and guide further model improvements.
- We observe accuracy of 41%
- Macro Average F1 score of 20%
- Weighted average F1 score of 29%
#call the defined helper function to display the heatmap of confusion_matrix
plot_confusion_matrix(y_test_classes,y_pred_classes)
Observations:TensorFlow's confusion matrix function provides the raw counts
- We observe that majority of the classes are not predicted correctly.
Model performance is not good
- In comparison to the rest, we can see that classes 0,1, 2, 4,7,9,11 are all not predicted correctly.
- We can also observe that classes 3,6 and 10 only are classified better
Model 5: Data Augmentation Model and Batch Normalization¶
# Clearing backend
backend.clear_session()
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
When building our custom model, we have used Batch Normalization and Dropout layers as regularization techniques to prevent overfitting.
# Intializing a sequential model
model5 = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model5.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model5.add(MaxPooling2D((2, 2)))
model5.add(BatchNormalization())
model5.add(Conv2D(32, (3, 3), activation='relu'))
model5.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model5.add(Flatten())
model5.add(Dense(16, activation='relu'))
model5.add(Dropout(0.3))
model5.add(Dense(12, activation='softmax'))
#Use Adam Optimizer
opt=Adam()
# Compile model
model5.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model5.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 62, 62, 64) 1792
max_pooling2d (MaxPooling2 (None, 31, 31, 64) 0
D)
batch_normalization (Batch (None, 31, 31, 64) 256
Normalization)
conv2d_1 (Conv2D) (None, 29, 29, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 14, 14, 32) 0
g2D)
flatten (Flatten) (None, 6272) 0
dense (Dense) (None, 16) 100368
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 121084 (472.98 KB)
Trainable params: 120956 (472.48 KB)
Non-trainable params: 128 (512.00 Byte)
_________________________________________________________________
Fitting the model on training data using ReduceLRonPlateau()¶
callbacks set
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.001)
start = time.time()
history_5 = model5.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
seed=42,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
callbacks=[reduce_lr],
verbose=2)
end = time.time()
Epoch 1/30 47/47 - 6s - loss: 2.2279 - accuracy: 0.2426 - val_loss: 2.4677 - val_accuracy: 0.2329 - lr: 0.0010 - 6s/epoch - 137ms/step Epoch 2/30 47/47 - 6s - loss: 2.1242 - accuracy: 0.2796 - val_loss: 2.4652 - val_accuracy: 0.1382 - lr: 0.0010 - 6s/epoch - 137ms/step Epoch 3/30 47/47 - 6s - loss: 2.0484 - accuracy: 0.3088 - val_loss: 2.4361 - val_accuracy: 0.2224 - lr: 0.0010 - 6s/epoch - 132ms/step Epoch 4/30 47/47 - 7s - loss: 1.9673 - accuracy: 0.3337 - val_loss: 2.4472 - val_accuracy: 0.1382 - lr: 0.0010 - 7s/epoch - 141ms/step Epoch 5/30 47/47 - 6s - loss: 1.8849 - accuracy: 0.3659 - val_loss: 2.3784 - val_accuracy: 0.2092 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 6/30 47/47 - 7s - loss: 1.8502 - accuracy: 0.3696 - val_loss: 2.3643 - val_accuracy: 0.2908 - lr: 0.0010 - 7s/epoch - 140ms/step Epoch 7/30 47/47 - 6s - loss: 1.7847 - accuracy: 0.4009 - val_loss: 2.2215 - val_accuracy: 0.3632 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 8/30 47/47 - 7s - loss: 1.7806 - accuracy: 0.3959 - val_loss: 2.0960 - val_accuracy: 0.3671 - lr: 0.0010 - 7s/epoch - 139ms/step Epoch 9/30 47/47 - 6s - loss: 1.7482 - accuracy: 0.3901 - val_loss: 2.0572 - val_accuracy: 0.3171 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 10/30 47/47 - 6s - loss: 1.7013 - accuracy: 0.4133 - val_loss: 1.9279 - val_accuracy: 0.4224 - lr: 0.0010 - 6s/epoch - 137ms/step Epoch 11/30 47/47 - 6s - loss: 1.6948 - accuracy: 0.4160 - val_loss: 1.8531 - val_accuracy: 0.4829 - lr: 0.0010 - 6s/epoch - 130ms/step Epoch 12/30 47/47 - 6s - loss: 1.6802 - accuracy: 0.4147 - val_loss: 1.7052 - val_accuracy: 0.4092 - lr: 0.0010 - 6s/epoch - 137ms/step Epoch 13/30 47/47 - 6s - loss: 1.6631 - accuracy: 0.4234 - val_loss: 1.8549 - val_accuracy: 0.3592 - lr: 0.0010 - 6s/epoch - 133ms/step Epoch 14/30 47/47 - 6s - loss: 1.6195 - accuracy: 0.4328 - val_loss: 1.8095 - val_accuracy: 0.4553 - lr: 0.0010 - 6s/epoch - 138ms/step Epoch 15/30 47/47 - 6s - loss: 1.6443 - accuracy: 0.4247 - val_loss: 1.5628 - val_accuracy: 0.4934 - lr: 0.0010 - 6s/epoch - 132ms/step Epoch 16/30 47/47 - 6s - loss: 1.5823 - accuracy: 0.4382 - val_loss: 1.4298 - val_accuracy: 0.5855 - lr: 0.0010 - 6s/epoch - 132ms/step Epoch 17/30 47/47 - 6s - loss: 1.6063 - accuracy: 0.4365 - val_loss: 1.4294 - val_accuracy: 0.5500 - lr: 0.0010 - 6s/epoch - 132ms/step Epoch 18/30 47/47 - 6s - loss: 1.5875 - accuracy: 0.4479 - val_loss: 1.4138 - val_accuracy: 0.5737 - lr: 0.0010 - 6s/epoch - 132ms/step Epoch 19/30 47/47 - 6s - loss: 1.5371 - accuracy: 0.4570 - val_loss: 1.4206 - val_accuracy: 0.5763 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 20/30 47/47 - 6s - loss: 1.5489 - accuracy: 0.4476 - val_loss: 1.3660 - val_accuracy: 0.5737 - lr: 0.0010 - 6s/epoch - 134ms/step Epoch 21/30 47/47 - 6s - loss: 1.5506 - accuracy: 0.4493 - val_loss: 1.6317 - val_accuracy: 0.4579 - lr: 0.0010 - 6s/epoch - 125ms/step Epoch 22/30 47/47 - 6s - loss: 1.5344 - accuracy: 0.4476 - val_loss: 1.3387 - val_accuracy: 0.5513 - lr: 0.0010 - 6s/epoch - 133ms/step Epoch 23/30 47/47 - 6s - loss: 1.5327 - accuracy: 0.4538 - val_loss: 1.4868 - val_accuracy: 0.4987 - lr: 0.0010 - 6s/epoch - 129ms/step Epoch 24/30 47/47 - 6s - loss: 1.5075 - accuracy: 0.4617 - val_loss: 1.4073 - val_accuracy: 0.5329 - lr: 0.0010 - 6s/epoch - 133ms/step Epoch 25/30 47/47 - 6s - loss: 1.4870 - accuracy: 0.4674 - val_loss: 1.4726 - val_accuracy: 0.5382 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 26/30 47/47 - 6s - loss: 1.5059 - accuracy: 0.4624 - val_loss: 1.3301 - val_accuracy: 0.5868 - lr: 0.0010 - 6s/epoch - 134ms/step Epoch 27/30 47/47 - 6s - loss: 1.4863 - accuracy: 0.4768 - val_loss: 1.3470 - val_accuracy: 0.5658 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 28/30 47/47 - 6s - loss: 1.4492 - accuracy: 0.4782 - val_loss: 1.6515 - val_accuracy: 0.4434 - lr: 0.0010 - 6s/epoch - 133ms/step Epoch 29/30 47/47 - 6s - loss: 1.4454 - accuracy: 0.4798 - val_loss: 1.2316 - val_accuracy: 0.6079 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 30/30 47/47 - 6s - loss: 1.4847 - accuracy: 0.4674 - val_loss: 1.2130 - val_accuracy: 0.6197 - lr: 0.0010 - 6s/epoch - 132ms/step
model5_time = end-start
print("Time taken in minutes ",model5_time/60)
Time taken in minutes 3.2623358964920044
Model Evaluation¶
#Plot of Model Loss
plot(history_5,'loss')
#plot of model accuracy
plot(history_5,'accuracy')
#Evaluate the model on test data
accuracy5 = model5.evaluate(X_test_normalized, y_test_encoded, verbose=2)
30/30 - 0s - loss: 1.2551 - accuracy: 0.6137 - 388ms/epoch - 13ms/step
#Predictions on test data
# Here we would get the output as probablities for each category
y_pred=model5.predict(X_test_normalized)
30/30 [==============================] - 0s 13ms/step
Model Performance and Confusion Matrix¶
# Model Performance on training data
model5_train_perf = model_performance_classification(model5,X_train_normalized,y_train_encoded)
print(model5_train_perf)
95/95 [==============================] - 1s 12ms/step Accuracy Recall Precision F1 Score 0 0.416447 0.416447 0.917016 0.437219
# Model Performance on validation data
model5_val_perf = model_performance_classification(model5,X_val_normalized,y_val_encoded)
print(model5_val_perf)
24/24 [==============================] - 0s 10ms/step Accuracy Recall Precision F1 Score 0 0.377632 0.377632 0.884441 0.396408
# Model Performance on testing data
model5_test_perf = model_performance_classification(model5,X_test_normalized,y_test_encoded)
print(model5_test_perf)
30/30 [==============================] - 0s 13ms/step Accuracy Recall Precision F1 Score 0 0.005263 0.005263 1.0 0.01002
We will use the argmax() function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes
# Obtaining the categorical values from y_test_encoded and y_pred
y_test_classes = np.argmax(y_test_encoded, axis=1)
y_pred_classes = np.argmax(y_pred, axis=1)
# Performance summary using Classification report
report5 = classification_report(y_test_classes, y_pred_classes,zero_division=1)
print(report5)
precision recall f1-score support
0 1.00 0.00 0.00 53
1 0.55 0.92 0.69 78
2 0.69 0.34 0.46 58
3 0.61 0.91 0.73 122
4 1.00 0.00 0.00 44
5 0.85 0.73 0.78 95
6 0.59 0.95 0.73 131
7 1.00 0.00 0.00 44
8 0.54 0.77 0.63 103
9 0.00 0.00 0.00 46
10 0.67 0.88 0.76 99
11 0.53 0.27 0.36 77
accuracy 0.61 950
macro avg 0.67 0.48 0.43 950
weighted avg 0.65 0.61 0.53 950
The classification report would help you determine how well the CNN model is identifying different plant species. By analyzing the metrics for each species, we can identify potential weaknesses (e.g., species that are frequently misclassified) and guide further model improvements.
- We observe accuracy of 61%
- Macro Average F1 score of 43%
- Weighted average F1 score of 53%
#call the defined helper function to display the heatmap of confusion_matrix
plot_confusion_matrix(y_test_classes,y_pred_classes)
Observations:TensorFlow's confusion matrix function provides the raw counts
- We observe accuracy of the model is 61% better than other data augmentation model.
- In comparison to the rest, we can see that classes 1,3,6,8 ,10 are well classified.
- 0,4,7,9 are not predictly correctly
Model 6: Data Augmentation Model to overcome imbalance problem¶
Remember, data augmentation should not be used in the validation/test data set.
Prepare augmented data for minority class¶
Identify Minority and Majority Classes and Apply Augmentations to Minority Class
# Assuming 'y_train' contains the labels for your training data
from collections import Counter
class_counts = Counter(y_train['Label'])
minority_classes = [cls for cls, count in class_counts.items() if count < 300]
majority_classes = [cls for cls, count in class_counts.items() if count >= 300]
print("Minority Classes:", minority_classes)
print("Majority Classes:", majority_classes)
Minority Classes: ['Cleavers', 'Sugar beet', 'Black-grass', 'Maize', 'Charlock', 'Shepherds Purse', 'Common wheat'] Majority Classes: ['Scentless Mayweed', 'Small-flowered Cranesbill', 'Loose Silky-bent', 'Fat Hen', 'Common Chickweed']
# Create boolean mask to subset X_train_normalized and y_train_encoded
minority_mask = y_train['Label'].isin(minority_classes)
majority_mask = y_train['Label'].isin(majority_classes)
# Use the boolean mask to extract minority and majority data
X_train_minority = X_train_normalized[minority_mask]
y_train_minority = y_train_encoded[minority_mask]
X_train_majority = X_train_normalized[majority_mask]
y_train_majority = y_train_encoded[majority_mask]
train_datagen_minority = ImageDataGenerator(
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
train_generator_minority = train_datagen_minority.flow(
X_train_minority, y_train_minority, batch_size=32, shuffle=False)
def combine_generators(gen1, gen2):
while True:
X1, y1 = next(gen1)
X2, y2 = next(gen2)
yield np.concatenate([X1, X2]), np.concatenate([y1, y2])
train_datagen_majority = ImageDataGenerator() # No augmentations for majority
train_generator_majority = train_datagen_majority.flow(
X_train_majority, y_train_majority, batch_size=32, shuffle=True)
train_generator = combine_generators(train_generator_minority, train_generator_majority)
# Clearing backend
backend.clear_session()
# Intializing a sequential model
model6 = Sequential()
# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model6.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of output of first conv layer
model6.add(MaxPooling2D((2, 2)))
model6.add(BatchNormalization())
model6.add(Conv2D(32, (3, 3), activation='relu'))
model6.add(MaxPooling2D((2, 2)))
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model6.add(Flatten())
model6.add(Dense(16, activation='relu'))
model6.add(Dropout(0.3))
model6.add(Dense(12, activation='softmax'))
# Adam as Optimizer
opt=Adam()
# Compile model
model6.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model6.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 62, 62, 64) 1792
max_pooling2d (MaxPooling2 (None, 31, 31, 64) 0
D)
batch_normalization (Batch (None, 31, 31, 64) 256
Normalization)
conv2d_1 (Conv2D) (None, 29, 29, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 14, 14, 32) 0
g2D)
flatten (Flatten) (None, 6272) 0
dense (Dense) (None, 16) 100368
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 121084 (472.98 KB)
Trainable params: 120956 (472.48 KB)
Non-trainable params: 128 (512.00 Byte)
_________________________________________________________________
Fitting the model on training data¶
# Epochs
epochs = 30
# Batch size
batch_size = 64
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.001)
# steps_per_epoch calculation:
total_samples = X_train_minority.shape[0] + X_train_majority.shape[0] # Total minority + majority
start = time.time()
history_6 = model6.fit(train_generator,
epochs=epochs,
steps_per_epoch=total_samples // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
callbacks=[reduce_lr],
verbose=2)
end = time.time()
Epoch 1/30 47/47 - 7s - loss: 2.4790 - accuracy: 0.1244 - val_loss: 2.4772 - val_accuracy: 0.1382 - lr: 0.0010 - 7s/epoch - 153ms/step Epoch 2/30 47/47 - 6s - loss: 2.4354 - accuracy: 0.1405 - val_loss: 2.4698 - val_accuracy: 0.1342 - lr: 0.0010 - 6s/epoch - 133ms/step Epoch 3/30 47/47 - 6s - loss: 2.3696 - accuracy: 0.1757 - val_loss: 2.4639 - val_accuracy: 0.1382 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 4/30 47/47 - 6s - loss: 2.2717 - accuracy: 0.2285 - val_loss: 2.4549 - val_accuracy: 0.0987 - lr: 0.0010 - 6s/epoch - 129ms/step Epoch 5/30 47/47 - 6s - loss: 2.2129 - accuracy: 0.2446 - val_loss: 2.4478 - val_accuracy: 0.1447 - lr: 0.0010 - 6s/epoch - 126ms/step Epoch 6/30 47/47 - 6s - loss: 2.2114 - accuracy: 0.2279 - val_loss: 2.4524 - val_accuracy: 0.1382 - lr: 0.0010 - 6s/epoch - 125ms/step Epoch 7/30 47/47 - 6s - loss: 2.1674 - accuracy: 0.2654 - val_loss: 2.4420 - val_accuracy: 0.1092 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 8/30 47/47 - 6s - loss: 2.1375 - accuracy: 0.2604 - val_loss: 2.4443 - val_accuracy: 0.1382 - lr: 0.0010 - 6s/epoch - 125ms/step Epoch 9/30 47/47 - 6s - loss: 2.1287 - accuracy: 0.2624 - val_loss: 2.4318 - val_accuracy: 0.1526 - lr: 0.0010 - 6s/epoch - 133ms/step Epoch 10/30 47/47 - 6s - loss: 2.0701 - accuracy: 0.2917 - val_loss: 2.4341 - val_accuracy: 0.1474 - lr: 0.0010 - 6s/epoch - 124ms/step Epoch 11/30 47/47 - 6s - loss: 2.0522 - accuracy: 0.3061 - val_loss: 2.4205 - val_accuracy: 0.1868 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 12/30 47/47 - 6s - loss: 2.0166 - accuracy: 0.3116 - val_loss: 2.4282 - val_accuracy: 0.1474 - lr: 0.0010 - 6s/epoch - 124ms/step Epoch 13/30 47/47 - 6s - loss: 1.9997 - accuracy: 0.3136 - val_loss: 2.1393 - val_accuracy: 0.3539 - lr: 0.0010 - 6s/epoch - 134ms/step Epoch 14/30 47/47 - 6s - loss: 1.9817 - accuracy: 0.3196 - val_loss: 1.9796 - val_accuracy: 0.4092 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 15/30 47/47 - 6s - loss: 1.9236 - accuracy: 0.3290 - val_loss: 1.9642 - val_accuracy: 0.3921 - lr: 0.0010 - 6s/epoch - 134ms/step Epoch 16/30 47/47 - 6s - loss: 1.8889 - accuracy: 0.3367 - val_loss: 1.7832 - val_accuracy: 0.4342 - lr: 0.0010 - 6s/epoch - 126ms/step Epoch 17/30 47/47 - 6s - loss: 1.8668 - accuracy: 0.3683 - val_loss: 1.9458 - val_accuracy: 0.4026 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 18/30 47/47 - 6s - loss: 1.8517 - accuracy: 0.3721 - val_loss: 1.8344 - val_accuracy: 0.4184 - lr: 0.0010 - 6s/epoch - 129ms/step Epoch 19/30 47/47 - 6s - loss: 1.8119 - accuracy: 0.3861 - val_loss: 1.7099 - val_accuracy: 0.4592 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 20/30 47/47 - 6s - loss: 1.8270 - accuracy: 0.3686 - val_loss: 1.8705 - val_accuracy: 0.3684 - lr: 0.0010 - 6s/epoch - 132ms/step Epoch 21/30 47/47 - 6s - loss: 1.8192 - accuracy: 0.3851 - val_loss: 1.8123 - val_accuracy: 0.4263 - lr: 0.0010 - 6s/epoch - 123ms/step Epoch 22/30 47/47 - 6s - loss: 1.7971 - accuracy: 0.3905 - val_loss: 1.7067 - val_accuracy: 0.4632 - lr: 0.0010 - 6s/epoch - 130ms/step Epoch 23/30 47/47 - 6s - loss: 1.7539 - accuracy: 0.3950 - val_loss: 1.9126 - val_accuracy: 0.3934 - lr: 0.0010 - 6s/epoch - 126ms/step Epoch 24/30 47/47 - 6s - loss: 1.7669 - accuracy: 0.4036 - val_loss: 1.7293 - val_accuracy: 0.4658 - lr: 0.0010 - 6s/epoch - 133ms/step Epoch 25/30 47/47 - 6s - loss: 1.7356 - accuracy: 0.4233 - val_loss: 2.0127 - val_accuracy: 0.3842 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 26/30 47/47 - 6s - loss: 1.7338 - accuracy: 0.4039 - val_loss: 1.7308 - val_accuracy: 0.4645 - lr: 0.0010 - 6s/epoch - 127ms/step Epoch 27/30 47/47 - 6s - loss: 1.7410 - accuracy: 0.4059 - val_loss: 1.7118 - val_accuracy: 0.4842 - lr: 0.0010 - 6s/epoch - 128ms/step Epoch 28/30 47/47 - 6s - loss: 1.6847 - accuracy: 0.4345 - val_loss: 1.8182 - val_accuracy: 0.4434 - lr: 0.0010 - 6s/epoch - 126ms/step Epoch 29/30 47/47 - 6s - loss: 1.6801 - accuracy: 0.4235 - val_loss: 1.7626 - val_accuracy: 0.4500 - lr: 0.0010 - 6s/epoch - 131ms/step Epoch 30/30 47/47 - 6s - loss: 1.6977 - accuracy: 0.4261 - val_loss: 1.7875 - val_accuracy: 0.4263 - lr: 0.0010 - 6s/epoch - 124ms/step
model6_time = end-start
print("Time taken in minutes ",model6_time/60)
Time taken in minutes 3.0379550496737164
Model Evaluation¶
#Plot of Model Loss
plot(history_6,'loss')
#plot of model accuracy
plot(history_6,'accuracy')
#Evaluate the model on test data
accuracy6 = model6.evaluate(X_test_normalized, y_test_encoded, verbose=2)
30/30 - 0s - loss: 2.4321 - accuracy: 0.1379 - 406ms/epoch - 14ms/step
#Predictions on test data
# Here we would get the output as probablities for each category
y_pred=model6.predict(X_test_normalized)
30/30 [==============================] - 0s 14ms/step
Model Performance and Confusion Matrix¶
# Model Performance on training data
model6_train_perf = model_performance_classification(model6,X_train_normalized,y_train_encoded)
print(model6_train_perf)
95/95 [==============================] - 1s 13ms/step Accuracy Recall Precision F1 Score 0 0.065789 0.065789 0.69452 0.078517
# Model Performance on validation data
model6_val_perf = model_performance_classification(model6,X_val_normalized,y_val_encoded)
print(model6_val_perf)
24/24 [==============================] - 0s 11ms/step Accuracy Recall Precision F1 Score 0 0.059211 0.059211 0.82648 0.066727
# Model Performance on testing data
model6_test_perf = model_performance_classification(model6,X_test_normalized,y_test_encoded)
print(model6_test_perf)
30/30 [==============================] - 0s 12ms/step Accuracy Recall Precision F1 Score 0 0.058947 0.058947 0.857293 0.067048
We will use the argmax() function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes
# Obtaining the categorical values from y_test_encoded and y_pred
y_test_classes = np.argmax(y_test_encoded, axis=1)
y_pred_classes = np.argmax(y_pred, axis=1)
# Performance summary using Classification report
report6 = classification_report(y_test_classes, y_pred_classes,zero_division=1)
print(report6)
precision recall f1-score support
0 0.48 0.21 0.29 53
1 0.41 0.64 0.50 78
2 0.82 0.24 0.37 58
3 0.53 0.79 0.64 122
4 1.00 0.00 0.00 44
5 0.48 0.41 0.44 95
6 0.54 0.93 0.69 131
7 0.60 0.34 0.43 44
8 0.56 0.29 0.38 103
9 1.00 0.00 0.00 46
10 0.51 0.75 0.60 99
11 0.49 0.49 0.49 77
accuracy 0.51 950
macro avg 0.62 0.42 0.40 950
weighted avg 0.58 0.51 0.46 950
The classification report would help you determine how well the CNN model is identifying different plant species. By analyzing the metrics for each species, we can identify potential weaknesses (e.g., species that are frequently misclassified) and guide further model improvements.
- We observe accuracy of 51%
- Macro Average F1 score of 40%
- Weighted average F1 score of 46%
#call the defined helper function to display the heatmap of confusion_matrix
plot_confusion_matrix(y_test_classes,y_pred_classes)
Observations:TensorFlow's confusion matrix function provides the raw counts
- We observe that majority of the classes are not predicted correctly.Model is performing poorly
- In comparison to the rest, we can see that classes 1, 3 ,6 and 10 are well classified.
- We can also observe that classes 0,5,6,11 are mostly misclassified.
- 4,9 are mostly always wrongly predicted
Final Model¶
Helper Functions for Model Performance Comparison¶
def get_accuracy_from_report(report):
"""
Extracts the accuracy value from a classification report string.
Args:
report: The classification report string.
Returns:
The accuracy value as a string, or None if not found.
"""
accuracy_str = None
# Find the line containing "accuracy"
accuracy_str = re.search(r"accuracy\s+([\d.]+)", report)
if accuracy_str: # Check if match found
accuracy_str = accuracy_str.group(1)
return accuracy_str
def get_precision_from_report(report,average):
"""
Extracts the precision value from a classification report string.
Args:
report: The classification report string.
Returns:
The precision value as a string, or None if not found.
"""
precision_str = None
if average == "weighted avg" :
# Find the line containing "weighted avg"
weighted_avg_line = re.search(r"weighted avg\s+([\d.]+)\s+([\d.]+)\s+([\d.]+)", report)
if weighted_avg_line: # Check if match found
precision_str = weighted_avg_line.group(1)
elif average == "macro avg" :
# Find the line containing "macro avg"
macro_avg_line = re.search(r"macro avg\s+([\d.]+)\s+([\d.]+)\s+([\d.]+)", report)
if macro_avg_line: # Check if match found
precision_str = macro_avg_line.group(1)
return precision_str
def get_recall_score_from_report(report,average):
"""
Extracts the recall-score value from a classification report string.
Args:
report: The classification report string.
Returns:
The recall-score value as a string, or None if not found.
"""
recall_str = None
if average == "weighted avg" :
# Find the line containing "weighted avg"
weighted_avg_line = re.search(r"weighted avg\s+([\d.]+)\s+([\d.]+)\s+([\d.]+)", report)
if weighted_avg_line: # Check if match found
recall_str = weighted_avg_line.group(2) # recall is the second group
elif average == "macro avg" :
# Find the line containing "macro avg"
macro_avg_line = re.search(r"macro avg\s+([\d.]+)\s+([\d.]+)\s+([\d.]+)", report)
if macro_avg_line: # Check if match found
recall_str = macro_avg_line.group(2) # recall is the second group
return recall_str
def get_f1_score_from_report(report,average):
"""
Extracts the f1-score value from a classification report string.
Args:
report: The classification report string.
Returns:
The f1-score value as a string, or None if not found.
"""
f1_score_str = None
if average == "weighted avg" :
# Find the line containing "weighted avg"
weighted_avg_line = re.search(r"weighted avg\s+([\d.]+)\s+([\d.]+)\s+([\d.]+)", report)
if weighted_avg_line: # Check if match found
f1_score_str = weighted_avg_line.group(3) # F1-score is the third group
elif average == "macro avg" :
# Find the line containing "macro avg"
macro_avg_line = re.search(r"macro avg\s+([\d.]+)\s+([\d.]+)\s+([\d.]+)", report)
if macro_avg_line: # Check if match found
f1_score_str = macro_avg_line.group(3) # F1-score is the third group
return f1_score_str
Model Performance Comparison¶
#models performance summary using history
model_accuracy = {
"Models" : ["Model 1","Model 2","Model 3","Model 4","Model 5","Model 6"],
"Model Description" : ["CNN Model",
"CNN Data Augmentation with Rotation",
"CNN Data Augmentation with more augmentation techniques",
"CNN Data Augmentation with ReduceLROnPlateau()",
"CNN Data Augmentation with ReduceLR and Batch Normalization",
"CNN Data Augmentation with ReduceLR - Augmentation on Imbalanced data" ],
"Train Accuracy" :[ #Final Training Accuracy
history_1.history['accuracy'][-1],
history_2.history['accuracy'][-1],
history_3.history['accuracy'][-1],
history_4.history['accuracy'][-1],
history_5.history['accuracy'][-1],
history_6.history['accuracy'][-1],
],
"Validation Accuracy" :[
history_1.history['val_accuracy'][-1],
history_2.history['val_accuracy'][-1],
history_3.history['val_accuracy'][-1],
history_4.history['val_accuracy'][-1],
history_5.history['val_accuracy'][-1],
history_6.history['val_accuracy'][-1],
],
"Test Accuracy" :[
accuracy1[1],
accuracy2[1],
accuracy3[1],
accuracy4[1],
accuracy5[1],
accuracy6[1],
],
}
print("Model Summary on Test Data using History/Accuracy:")
model_df = pd.DataFrame(model_accuracy)
model_df.T
Model Summary on Test Data using History/Accuracy:
| 0 | 1 | 2 | 3 | 4 | 5 | |
|---|---|---|---|---|---|---|
| Models | Model 1 | Model 2 | Model 3 | Model 4 | Model 5 | Model 6 |
| Model Description | CNN Model | CNN Data Augmentation with Rotation | CNN Data Augmentation with more augmentation t... | CNN Data Augmentation with ReduceLROnPlateau() | CNN Data Augmentation with ReduceLR and Batch ... | CNN Data Augmentation with ReduceLR - Augmenta... |
| Train Accuracy | 0.469737 | 0.455645 | 0.410282 | 0.333669 | 0.467406 | 0.426075 |
| Validation Accuracy | 0.576316 | 0.584211 | 0.563158 | 0.4 | 0.619737 | 0.426316 |
| Test Accuracy | 0.608421 | 0.586316 | 0.572632 | 0.410526 | 0.613684 | 0.137895 |
- Model 5 CNN Data Augmentation with reduceLROnPlateau() and BatchNormalization() performs better than other models
with accuracy of 61.97 % on validation data and 61.4% on Testing data and 46.7% on Training
#models performance summary on test data using classification report
model_data = {
"Models" : ["Model 1","Model 2","Model 3","Model 4","Model 5","Model 6"],
"Model Description" : ["CNN Model",
"CNN Data Augmentation with Rotation",
"CNN Data Augmentation with more augmentation techniques",
"CNN Data Augmentation with ReduceLROnPlateau()",
"CNN Data Augmentation with ReduceLR and Batch Normalization",
"CNN Data Augmentation with ReduceLR - Augmentation on Imbalanced data" ],
"Accuracy" :[
accuracy1[1],
accuracy2[1],
accuracy3[1],
accuracy4[1],
accuracy5[1],
accuracy6[1],
],
"Loss" :[
accuracy1[0],
accuracy2[0],
accuracy3[0],
accuracy4[0],
accuracy5[0],
accuracy6[0],
],
"Classification Report Accuracy" :[
get_accuracy_from_report(report1),
get_accuracy_from_report(report2),
get_accuracy_from_report(report3),
get_accuracy_from_report(report4),
get_accuracy_from_report(report5),
get_accuracy_from_report(report6),
],
"Macro Avg Precision" :[
get_precision_from_report(report1,'macro avg'),
get_precision_from_report(report2,'macro avg'),
get_precision_from_report(report3,'macro avg'),
get_precision_from_report(report4,'macro avg'),
get_precision_from_report(report5,'macro avg'),
get_precision_from_report(report6,'macro avg'),
],
"Weighted Avg Precision" :[
get_precision_from_report(report1,'weighted avg'),
get_precision_from_report(report2,'weighted avg'),
get_precision_from_report(report3,'weighted avg'),
get_precision_from_report(report4,'weighted avg'),
get_precision_from_report(report5,'weighted avg'),
get_precision_from_report(report6,'weighted avg'),
],
"Macro Avg Recall" :[
get_recall_score_from_report(report1,'macro avg'),
get_recall_score_from_report(report2,'macro avg'),
get_recall_score_from_report(report3,'macro avg'),
get_recall_score_from_report(report4,'macro avg'),
get_recall_score_from_report(report5,'macro avg'),
get_recall_score_from_report(report6,'macro avg'),
],
"Weighted Avg Recall" :[
get_recall_score_from_report(report1,'weighted avg'),
get_recall_score_from_report(report2,'weighted avg'),
get_recall_score_from_report(report3,'weighted avg'),
get_recall_score_from_report(report4,'weighted avg'),
get_recall_score_from_report(report5,'weighted avg'),
get_recall_score_from_report(report6,'weighted avg'),
],
"Macro Avg F1-Score" :[
get_f1_score_from_report(report1,'macro avg'),
get_f1_score_from_report(report2,'macro avg'),
get_f1_score_from_report(report3,'macro avg'),
get_f1_score_from_report(report4,'macro avg'),
get_f1_score_from_report(report5,'macro avg'),
get_f1_score_from_report(report6,'macro avg'),
],
"Macro Avg F1-Score" :[
get_f1_score_from_report(report1,'macro avg'),
get_f1_score_from_report(report2,'macro avg'),
get_f1_score_from_report(report3,'macro avg'),
get_f1_score_from_report(report4,'macro avg'),
get_f1_score_from_report(report5,'macro avg'),
get_f1_score_from_report(report6,'macro avg'),
],
"Weighted Avg F1-Score":[
get_f1_score_from_report(report1,'weighted avg'),
get_f1_score_from_report(report2,'weighted avg'),
get_f1_score_from_report(report3,'weighted avg'),
get_f1_score_from_report(report4,'weighted avg'),
get_f1_score_from_report(report5,'weighted avg'),
get_f1_score_from_report(report6,'weighted avg'),
],
"Time (seconds)" :[
model1_time,
model2_time,
model3_time,
model4_time,
model5_time,
model6_time,
]
}
print("Model Summary on Test Data using Classification Report:")
model_data_df = pd.DataFrame(model_data)
model_data_df.T
Model Summary on Test Data using Classification Report:
| 0 | 1 | 2 | 3 | 4 | 5 | |
|---|---|---|---|---|---|---|
| Models | Model 1 | Model 2 | Model 3 | Model 4 | Model 5 | Model 6 |
| Model Description | CNN Model | CNN Data Augmentation with Rotation | CNN Data Augmentation with more augmentation t... | CNN Data Augmentation with ReduceLROnPlateau() | CNN Data Augmentation with ReduceLR and Batch ... | CNN Data Augmentation with ReduceLR - Augmenta... |
| Accuracy | 0.608421 | 0.586316 | 0.572632 | 0.410526 | 0.613684 | 0.137895 |
| Loss | 1.20045 | 1.146854 | 1.359193 | 1.659105 | 1.25509 | 2.43206 |
| Classification Report Accuracy | 0.61 | 0.59 | 0.57 | 0.41 | 0.61 | 0.51 |
| Macro Avg Precision | 0.63 | 0.69 | 0.66 | 0.75 | 0.67 | 0.62 |
| Weighted Avg Precision | 0.63 | 0.65 | 0.63 | 0.65 | 0.65 | 0.58 |
| Macro Avg Recall | 0.49 | 0.47 | 0.45 | 0.29 | 0.48 | 0.42 |
| Weighted Avg Recall | 0.61 | 0.59 | 0.57 | 0.41 | 0.61 | 0.51 |
| Macro Avg F1-Score | 0.45 | 0.43 | 0.40 | 0.20 | 0.43 | 0.40 |
| Weighted Avg F1-Score | 0.55 | 0.52 | 0.50 | 0.29 | 0.53 | 0.46 |
| Time (seconds) | 151.713536 | 183.132764 | 189.401192 | 208.650388 | 195.740154 | 182.277303 |
-Based on the above comparison on the performance of the model , Model 5 - CNN Model - Data Augmentation with ReductionLR and Batch Normalization has better performance than other models .
- Model 5 has test accuracy of 61.4% and Model 1 is next good performance of 60.8% accuracy
Other models have lesser accuracy
- Model 5 has macro avg F1 score of 43% ,
- Model 5 has weighted avg F1 score of 53%,
We will choose Model 5 as the best model
Final Model Selection¶
- Model 5 is chosen as best model
Comment on the final model you have selected and use the same in the below code to visualize the image.
Visualizing the prediction¶
Assign the best chosen model and use it to visualise the prediction
new_model = model5
# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[22])
plt.show()
print('Predicted Label', enc.inverse_transform(new_model.predict((X_test_normalized[22].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[22]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[37])
plt.show()
print('Predicted Label', enc.inverse_transform(new_model.predict((X_test_normalized[33].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[78])
plt.show()
print('Predicted Label', enc.inverse_transform(new_model.predict((X_test_normalized[78].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[78]) # using inverse_transform() to get the output label from the output vector
1/1 [==============================] - 0s 20ms/step Predicted Label ['Small-flowered Cranesbill'] True Label Common Chickweed
1/1 [==============================] - 0s 18ms/step Predicted Label ['Charlock'] True Label Charlock
1/1 [==============================] - 0s 19ms/step Predicted Label ['Loose Silky-bent'] True Label Loose Silky-bent
Actionable Insights and Business Recommendations¶
As we have seen, the CNN model with data-augmentation techiquest, reduceLROnPlateau() and Batch Normalization was able to predict the test image correctly with a test accuracy of 61.4%.
There are still scope for improvement in the test accuracy of the CNN model chosen here. Different architectures and optimizers can be used to built to obtain a better plant seed classifier.
Transfer learning can be applied to the dataset to improve the accuracy.
Once the desired performance is achieved from the model, the company can use it to classify different images being uploaded to the website.
For this project , we will choose model 5 that has the applied Batch Normalization techniques ,dataaugmentation rotation of images, and reduce learning rate callback parameter will be used and shared with the business.
Scope of Improvement¶
- These models can be further improved by training with different filter sizes and different number of filters.
- These models can also be trained on the original image_size i.e 128 x 128 rather than being reduced to 64.
- Data Augmentation can be performed more and dropout_rate can be changed to improve the model performance.
- Transfer Learning architectures can be used to train the CNN model to improve the performance and reap the benefits of Transfer Learning - less compute ,trained well even when input dataset is less. Pre-built architecture models like VGG16, can be used as that was trained on the ImageNet dataset and is the runner-up in the ImageNet competition in 2014. For training VGG16, directly use the convolutional and pooling layers and freeze their weights i.e. no training will be done on them. For classification, replace the existing fully-connected layers with FC layers created specifically for our problem.
Appendix: Transfer Learning using VGG16¶
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
from keras.applications.vgg16 import VGG16
vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (64,64,3))
vgg_model.summary()
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 64, 64, 3)] 0
block1_conv1 (Conv2D) (None, 64, 64, 64) 1792
block1_conv2 (Conv2D) (None, 64, 64, 64) 36928
block1_pool (MaxPooling2D) (None, 32, 32, 64) 0
block2_conv1 (Conv2D) (None, 32, 32, 128) 73856
block2_conv2 (Conv2D) (None, 32, 32, 128) 147584
block2_pool (MaxPooling2D) (None, 16, 16, 128) 0
block3_conv1 (Conv2D) (None, 16, 16, 256) 295168
block3_conv2 (Conv2D) (None, 16, 16, 256) 590080
block3_conv3 (Conv2D) (None, 16, 16, 256) 590080
block3_pool (MaxPooling2D) (None, 8, 8, 256) 0
block4_conv1 (Conv2D) (None, 8, 8, 512) 1180160
block4_conv2 (Conv2D) (None, 8, 8, 512) 2359808
block4_conv3 (Conv2D) (None, 8, 8, 512) 2359808
block4_pool (MaxPooling2D) (None, 4, 4, 512) 0
block5_conv1 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv2 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv3 (Conv2D) (None, 4, 4, 512) 2359808
block5_pool (MaxPooling2D) (None, 2, 2, 512) 0
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
layer.trainable = False
#check all are non trainable params for VGG model
vgg_model.summary()
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 64, 64, 3)] 0
block1_conv1 (Conv2D) (None, 64, 64, 64) 1792
block1_conv2 (Conv2D) (None, 64, 64, 64) 36928
block1_pool (MaxPooling2D) (None, 32, 32, 64) 0
block2_conv1 (Conv2D) (None, 32, 32, 128) 73856
block2_conv2 (Conv2D) (None, 32, 32, 128) 147584
block2_pool (MaxPooling2D) (None, 16, 16, 128) 0
block3_conv1 (Conv2D) (None, 16, 16, 256) 295168
block3_conv2 (Conv2D) (None, 16, 16, 256) 590080
block3_conv3 (Conv2D) (None, 16, 16, 256) 590080
block3_pool (MaxPooling2D) (None, 8, 8, 256) 0
block4_conv1 (Conv2D) (None, 8, 8, 512) 1180160
block4_conv2 (Conv2D) (None, 8, 8, 512) 2359808
block4_conv3 (Conv2D) (None, 8, 8, 512) 2359808
block4_pool (MaxPooling2D) (None, 4, 4, 512) 0
block5_conv1 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv2 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv3 (Conv2D) (None, 4, 4, 512) 2359808
block5_pool (MaxPooling2D) (None, 2, 2, 512) 0
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 0 (0.00 Byte)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
# Intializing a sequential model
model= Sequential()
# Adding the convolutional part of the VGG16 model from above
model.add(vgg_model)
# Flattening the output of the VGG16 model because it is from a convolutional layer
model.add(Flatten())
# Adding a fully connected dense layer
model.add(Dense(16, activation='relu'))
model.add(Dropout(0.3))
model.add(Dense(12, activation='softmax'))
# Adam as Optimizer
opt=Adam()
# Compile model
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 2, 2, 512) 14714688
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 16) 32784
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 14747676 (56.26 MB)
Trainable params: 32988 (128.86 KB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
Fitting the model on training data¶
# Data generator with rotation set
train_datagen = ImageDataGenerator(
horizontal_flip = True,
height_shift_range= 0.1,
width_shift_range=0.1,
rotation_range=20,
fill_mode='nearest')
# Epochs
epochs = 30
# Batch size
batch_size = 64
augmented_data = train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
seed=42,
shuffle=False)
start = time.time()
history = model.fit(augmented_data,
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=2)
end = time.time()
Epoch 1/30 47/47 - 26s - loss: 2.3741 - accuracy: 0.1851 - val_loss: 2.2333 - val_accuracy: 0.2684 - 26s/epoch - 553ms/step Epoch 2/30 47/47 - 25s - loss: 2.2179 - accuracy: 0.2480 - val_loss: 2.1104 - val_accuracy: 0.3329 - 25s/epoch - 529ms/step Epoch 3/30 47/47 - 26s - loss: 2.1321 - accuracy: 0.2698 - val_loss: 2.0070 - val_accuracy: 0.3605 - 26s/epoch - 551ms/step Epoch 4/30 47/47 - 25s - loss: 2.0580 - accuracy: 0.2886 - val_loss: 1.9384 - val_accuracy: 0.3592 - 25s/epoch - 533ms/step Epoch 5/30 47/47 - 25s - loss: 1.9577 - accuracy: 0.3205 - val_loss: 1.8319 - val_accuracy: 0.3908 - 25s/epoch - 530ms/step Epoch 6/30 47/47 - 25s - loss: 1.9130 - accuracy: 0.3320 - val_loss: 1.7925 - val_accuracy: 0.4066 - 25s/epoch - 532ms/step Epoch 7/30 47/47 - 25s - loss: 1.8406 - accuracy: 0.3545 - val_loss: 1.7275 - val_accuracy: 0.4197 - 25s/epoch - 528ms/step Epoch 8/30 47/47 - 25s - loss: 1.8117 - accuracy: 0.3622 - val_loss: 1.6872 - val_accuracy: 0.4316 - 25s/epoch - 523ms/step Epoch 9/30 47/47 - 25s - loss: 1.7866 - accuracy: 0.3669 - val_loss: 1.6645 - val_accuracy: 0.4539 - 25s/epoch - 523ms/step Epoch 10/30 47/47 - 25s - loss: 1.7351 - accuracy: 0.3847 - val_loss: 1.6291 - val_accuracy: 0.4592 - 25s/epoch - 523ms/step Epoch 11/30 47/47 - 24s - loss: 1.7197 - accuracy: 0.3948 - val_loss: 1.6034 - val_accuracy: 0.4658 - 24s/epoch - 519ms/step Epoch 12/30 47/47 - 24s - loss: 1.7169 - accuracy: 0.3844 - val_loss: 1.5961 - val_accuracy: 0.4487 - 24s/epoch - 518ms/step Epoch 13/30 47/47 - 24s - loss: 1.6750 - accuracy: 0.3908 - val_loss: 1.5736 - val_accuracy: 0.4671 - 24s/epoch - 508ms/step Epoch 14/30 47/47 - 24s - loss: 1.6560 - accuracy: 0.4163 - val_loss: 1.5237 - val_accuracy: 0.5013 - 24s/epoch - 521ms/step Epoch 15/30 47/47 - 25s - loss: 1.6546 - accuracy: 0.4089 - val_loss: 1.5107 - val_accuracy: 0.5171 - 25s/epoch - 523ms/step Epoch 16/30 47/47 - 25s - loss: 1.6367 - accuracy: 0.4130 - val_loss: 1.5253 - val_accuracy: 0.4947 - 25s/epoch - 526ms/step Epoch 17/30 47/47 - 25s - loss: 1.6233 - accuracy: 0.4217 - val_loss: 1.5078 - val_accuracy: 0.5053 - 25s/epoch - 525ms/step Epoch 18/30 47/47 - 24s - loss: 1.6041 - accuracy: 0.4274 - val_loss: 1.5036 - val_accuracy: 0.5013 - 24s/epoch - 519ms/step Epoch 19/30 47/47 - 25s - loss: 1.5839 - accuracy: 0.4399 - val_loss: 1.4613 - val_accuracy: 0.5105 - 25s/epoch - 522ms/step Epoch 20/30 47/47 - 25s - loss: 1.5872 - accuracy: 0.4254 - val_loss: 1.4470 - val_accuracy: 0.5197 - 25s/epoch - 529ms/step Epoch 21/30 47/47 - 24s - loss: 1.5699 - accuracy: 0.4254 - val_loss: 1.4320 - val_accuracy: 0.5132 - 24s/epoch - 521ms/step Epoch 22/30 47/47 - 25s - loss: 1.5653 - accuracy: 0.4355 - val_loss: 1.4343 - val_accuracy: 0.5276 - 25s/epoch - 529ms/step Epoch 23/30 47/47 - 25s - loss: 1.5716 - accuracy: 0.4301 - val_loss: 1.4224 - val_accuracy: 0.5237 - 25s/epoch - 524ms/step Epoch 24/30 47/47 - 24s - loss: 1.5579 - accuracy: 0.4365 - val_loss: 1.4491 - val_accuracy: 0.5066 - 24s/epoch - 517ms/step Epoch 25/30 47/47 - 25s - loss: 1.5272 - accuracy: 0.4489 - val_loss: 1.4009 - val_accuracy: 0.5382 - 25s/epoch - 523ms/step Epoch 26/30 47/47 - 25s - loss: 1.5385 - accuracy: 0.4419 - val_loss: 1.3992 - val_accuracy: 0.5329 - 25s/epoch - 527ms/step Epoch 27/30 47/47 - 25s - loss: 1.5325 - accuracy: 0.4546 - val_loss: 1.3833 - val_accuracy: 0.5211 - 25s/epoch - 522ms/step Epoch 28/30 47/47 - 25s - loss: 1.5275 - accuracy: 0.4496 - val_loss: 1.3675 - val_accuracy: 0.5421 - 25s/epoch - 524ms/step Epoch 29/30 47/47 - 25s - loss: 1.5037 - accuracy: 0.4614 - val_loss: 1.3994 - val_accuracy: 0.5237 - 25s/epoch - 527ms/step Epoch 30/30 47/47 - 25s - loss: 1.4999 - accuracy: 0.4469 - val_loss: 1.3822 - val_accuracy: 0.5224 - 25s/epoch - 525ms/step
model_time = end-start
print("Time taken in minutes ",model_time/60)
Time taken in minutes 15.051047666867573
Model Evaluation¶
#Plot of Model Loss
plot(history,'loss')
#plot of model accuracy
plot(history,'accuracy')
#Evaluate the model on test data
#model.evaluate() is used to evaluate the model's performance on the test dataset (X_test_normalized, y_test_encoded). This is the correct approach to get a final, unbiased assessment of the trained model
accuracy = model.evaluate(X_test_normalized, y_test_encoded, verbose=2)
30/30 - 6s - loss: 1.3973 - accuracy: 0.5147 - 6s/epoch - 206ms/step
#Predictions on test data
# Here we would get the output as probablities for each category
y_pred=model.predict(X_test_normalized)
30/30 [==============================] - 6s 197ms/step
Model Performance and Confusion Matrix¶
# Model Performance on training data
model_train_perf = model_performance_classification(model,X_train_normalized,y_train_encoded)
print(model_train_perf)
95/95 [==============================] - 19s 200ms/step Accuracy Recall Precision F1 Score 0 0.267434 0.267434 0.852746 0.328006
# Model Performance on validation data
model_val_perf = model_performance_classification(model,X_val_normalized,y_val_encoded)
print(model_val_perf)
24/24 [==============================] - 5s 206ms/step Accuracy Recall Precision F1 Score 0 0.218421 0.218421 0.819904 0.263313
# Model Performance on testing data
model_test_perf = model_performance_classification(model,X_test_normalized,y_test_encoded)
print(model_test_perf)
30/30 [==============================] - 6s 196ms/step Accuracy Recall Precision F1 Score 0 0.230526 0.230526 0.84502 0.282303
We will use the argmax() function to obtain the maximum value over each category on both y_test_encoded and y_pred and obtain their respective classes
# Obtaining the categorical values from y_test_encoded and y_pred
y_test_classes = np.argmax(y_test_encoded, axis=1)
y_pred_classes = np.argmax(y_pred, axis=1)
# Performance summary using Classification report
report = classification_report(y_test_classes, y_pred_classes,zero_division=1)
print(report)
precision recall f1-score support
0 0.48 0.21 0.29 53
1 0.41 0.64 0.50 78
2 0.82 0.24 0.37 58
3 0.53 0.79 0.64 122
4 1.00 0.00 0.00 44
5 0.48 0.41 0.44 95
6 0.54 0.93 0.69 131
7 0.60 0.34 0.43 44
8 0.56 0.29 0.38 103
9 1.00 0.00 0.00 46
10 0.51 0.75 0.60 99
11 0.49 0.49 0.49 77
accuracy 0.51 950
macro avg 0.62 0.42 0.40 950
weighted avg 0.58 0.51 0.46 950
The classification report would help you determine how well the CNN model is identifying different plant species. By analyzing the metrics for each species, we can identify potential weaknesses (e.g., species that are frequently misclassified) and guide further model improvements.
Test Data Performance
- We observe accuracy of 51%
- Macro Average F1 score of 40%
- Weighted average F1 score of 46%
#call the defined helper function to display the heatmap of confusion_matrix
plot_confusion_matrix(y_test_classes,y_pred_classes)
Observations:TensorFlow's confusion matrix function provides the raw counts
- We observe that majority of the classes are not predicted correctly.
- VGG16 transfer learning has accuracy of 51% , Model can be fine tuned further by varying parameters,epochs,batch size ,different augmentation
- In comparison to the rest, we can see that classes 3 ,6 are classified better.
- 0 ,4 and 9 are mostly wrongly predicted